Pattern recognizing apparatus and method

ABSTRACT

An environment recognizing unit extracts the first through N-th states from an input image and calls data corresponding to the first through N-th states from the first through N-th pattern recognizing units to perform a recognizing unit.

This application is a divisional of application Ser. No. 08/778,621,filed Jan. 3, 1997 now U.S. Pat. No. 6,104,833, now allowed.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an apparatus and method for recognizinga pattern, and realizes to recognize characters, graphics, and symbolscorrectly depending on various states of input images when used with aprinted character recognizing apparatus and a graphics recognizingapparatus as well as a handwritten character recognizing apparatus.

2. Description of the Related Art

Conventional handwritten character recognizing apparatuses such as anoptical character reader (OCR) are designed to automatically readingcharacters written on an accounting list, etc. and automaticallyinputting the characters to eliminate the necessity of manually findingcharacters written on the accounting list, etc. and inputting thecharacters through a keyboard.

FIG. 1 is a block diagram showing the configuration of the conventionalhandwritten character recognizing apparatus.

In FIG. 1, a form/document 311 is read using a scanner to obtain amultiple-value image of the form/document 311.

A preprocessing unit 312 binarizes a multiple-value image, removesnoises, and amends the position of the form/document 311.

Then, a character detecting unit 313 detects each character according toinformation about preliminarily defined ruled lines and positionalinformation about a character.

A character recognizing unit 314 recognizes each character and outputs acharacter code. The character is recognized by collating each feature ofan unknown character pattern detected by the character detecting unit313 with the feature of each character category preliminarily entered ina recognizing dictionary 315.

For example, a distance between feature vectors in a feature space iscomputed by converting a 2-dimensional character pattern into a featurevector in a feature space representing the feature of the character, asa similarity between the unknown character pattern and the charactercategory preliminarily entered in the recognizing dictionary 315. Whenthe shortest distance is obtained between the feature vector of theunknown character pattern and the feature vector of the charactercategory preliminarily entered in the recognizing dictionary 315, thecharacter category is recognized corresponding to the unknown characterpattern.

A threshold is set for a distance between two feature vectors to avoidmistakenly recognizing a non-character such as a deletion line, a noise,a symbol, etc. for a character and outputting a character code for anon-character. If the distance between the two feature vectors is largerthan the threshold, a reject code is output by determining that theunknown character pattern has no corresponding character categorypreliminarily entered in the recognizing dictionary 315, or that theunknown character pattern refers to a non-character.

The recognizing dictionary 315 also contains the features of thecharacter categories of high-quality characters, obscure characters, anddeformed characters. A high-quality character recognizing dictionary 315is referred to for high quality characters. An obscure characterrecognizing dictionary 315 is referred to for obscure characters. Adeformed-character recognizing dictionary 315 is referred to fordeformed characters. Thus, the difference in quality of the charactersin the form/document 311 can be processed correspondingly.

FIG. 2 shows the configuration of the character recognizing apparatusfor recognizing a character with a deletion line.

The character recognizing apparatus shown in FIG. 2 comprises an imageinput unit 491 for inputting an original image containing a characterand detecting or preprocessing a character from the input image, and anidentifying unit 492 for identifying a character by extracting thefeature of the character and comparing the extracted feature with thefeature of the standard pattern stored in the recognizing dictionary.

When a character mistakenly entered in a form is removed with a deletionline, for example, six or more horizontal lines are entered on thecharacter. It is determined that the character provided with six or morehorizontal lines cannot be identified, and the character is rejected bythe identifying unit 492 because it does not match any standard patternstored in the recognizing dictionary.

However, the handwritten character recognizing apparatus shown in FIG. 1equally processes a detected character among obscure characters,deformed characters, high-quality characters using the same recognizingdictionary 315.

Accordingly, there has been a problem that information about an obscurecharacter entered in the recognizing dictionary 315 has a bad influenceon the high-quality character recognizing process, and the obscurecharacter entered in the recognizing dictionary 315 prevents highquality characters from being successful read.

In addition to obscure and deformed states, there are variousenvironments for characters. For example, a character may touch itscharacter box. When a single recognizing dictionary 315 is referred toin various environments, they affect each other, thereby generating aproblem that the recognizing process cannot be performed with enhancedprecision.

When the character recognizing apparatus shown in FIG. 2 recognizes acharacter, six or more horizontal lines are required to delete anentered character using a deletion line. This is a heavy load to a userand therefore cannot be completely observed. As a result, a characterwith an apparent deletion line makes a small distance from a standardpattern stored in the recognizing dictionary and fails to be clearlydistinguished from a character without a deletion line. Thus, thecharacter to be deleted cannot be rejected and mistakenly read.

For example, as indicated by (A) shown in FIG. 3, the ‘0’ to be deletedis not rejected but recognized as ‘8’. As indicated by (B) shown in FIG.3, the ‘1’ to be deleted is not rejected but recognized as ‘8’. Asindicated by (C) shown in FIG. 3, the ‘7’ to be deleted is not rejectedbut recognized as ‘4’. As indicated by (D) shown in FIG. 3, the ‘6’ tobe deleted is not rejected but recognized as ‘6’.

SUMMARY OF THE INVENTION

The present invention aims at providing a pattern recognizing apparatusand method capable of appropriately recognizing a character with highprecision depending on the environment of the character.

According to the feature of the present invention, an input pattern isrecognized by extracting the first predetermined feature from the inputpattern and extracting the second predetermined feature from the inputpattern from which the first feature has been extracted.

As a result, a recognizing process can be performed depending on eachenvironment of a character.

According to other features of the present invention, a pattern isrecognized by extracting the state of a process object from an inputimage and selecting a recognizing process suitable for the state foreach process object.

Thus, a pattern recognizing process can be performed appropriately foreach state on the input image having various states, thereby realizingthe recognizing process with high precision.

According to other feature of the present invention, a state of aprocess object is extracted from an input image, and a patternrecognizing process exclusively for the first state is performed on theprocess object in the first state, and a pattern recognizing processexclusively for the second state is performed on the process object inthe second state.

Thus, the recognizing process on the process object in the first stateinteract with the recognizing process on the process object in thesecond state, thereby successfully performing the recognizing processeswith high precision.

According to other feature of the present invention, recognizingdictionaries are appropriately selected for an input image in variousstates.

For example, even if obscure characters, deformed characters, andhigh-quality characters are mixed in the input image, the recognizingprocess can be performed with high precision by using an obscurecharacter recognizing dictionary for obscure characters, adeformed-character recognizing dictionary for deformed characters, andhigh-quality character recognizing dictionary for high-qualitycharacters.

According to other feature of the present invention, identificationfunctions are appropriately selected for an input image in variousstates.

The recognizing process can be performed with high precision by, forexample, recognizing a character using a city block distance on acharacter written in a one-character box, and recognizing a characterusing a discriminant function on a character written in a free-pitch boxin consideration of the character detection reliability.

According to other feature of the present invention, knowledge isappropriately selected for an input image in various states.

The recognizing process can be performed with high precision by, forexample, setting a correspondence between an unknown character and acharacter category by dividing a character into character segments whenan unknown character is considerably deformed and has no correspondencewith a character category stored in the recognizing dictionary,computing the detection reliability using a discriminant functiongenerated based on a learning pattern when a character is detected froma character string, and evaluating the recognition reliability on abox-touching character using the reliability obtained through a learningpattern when the box-touching character is recognized.

According to other feature of the present invention, the recognizingprocess is performed according to priority until the reliability of therecognizing process reaches a predetermined value when a plurality ofrecognizing processes are called for a specified process object.

Thus, the reliability of the recognizing process can be enhanced and theprecision of the process can be successfully improved.

According to other feature of the present invention, a non-character isextracted from an input image and a non-character recognizing processand a character recognizing process are performed separately on theextracted non-character.

As a result, the recognizing process can be performed with highprecision with less characters mistaken for non-characters and with lessnon-characters mistaken for characters.

According to other feature of the present invention, the firstpredetermined feature is extracted from an input pattern, and the inputpattern is recognized by extracting the second predetermined featurefrom the input pattern from which the first predetermined feature hasnot been extracted.

Thus, a character with a deletion line can be distinguished from acharacter without a deletion line, and only the character without adeletion line can be recognized. Therefore, it is possible to prevent acharacter with a deletion line from being mistakenly recognized for anyother character.

According to other feature of the present invention, the firstpredetermined feature is extracted from an input pattern, a portioncontributing to the first predetermined feature can be removed from theinput pattern from which the first predetermined feature has beenextracted, and the input pattern is recognized based on a pattern fromwhich the portion contributing to the first predetermined feature hasbeen removed.

Therefore, only a deletion line can be removed from the character withthe deletion line when the character is recognized, thereby improvingthe precision in recognizing the character.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing the configuration of the conventionalcharacter recognizing apparatus;

FIG. 2 is a block diagram showing the configuration of the conventionalcharacter recognizing apparatus;

FIG. 3 shows an example of recognizing a character with a deletion line;

FIG. 4 is a block diagram showing the configuration of the patternrecognizing apparatus according to the first embodiment of the presentinvention;

FIG. 5 is a block diagram showing the functions of the patternrecognizing apparatus according to the second embodiment of the presentinvention;

FIG. 6 is a block diagram showing an embodiment of the practicalconfiguration of the environment recognizing unit shown in FIG. 5;

FIG. 7 is a block diagram showing an embodiment of the practicalconfiguration of the pattern recognizing apparatus shown in FIG. 5;

FIG. 8 is a flowchart showing an embodiment of the entire operations ofthe environment recognizing system shown in FIG. 1;

FIG. 9 is a flowchart showing an embodiment of the operations of thepreprocessing unit shown in FIG. 8;

FIG. 10 is a flowchart showing an embodiment of the operations of thelayout analyzing unit shown in FIG. 8;

FIG. 11 is a flowchart showing an embodiment of the operations of thequality analyzing unit shown in FIG. 8;

FIG. 12 is a flowchart showing an embodiment of the operations of thecorrection analyzing unit shown in FIG. 8;

FIG. 13 is a flowchart showing an embodiment of the operations of thecontrol unit for controlling character/non-character recognizingprocesses shown in FIG. 8;

FIG. 14 is a block diagram showing the configuration of the patternrecognizing apparatus according to an embodiment of the presentinvention;

FIG. 15 is a block diagram showing a practical configuration of thepattern recognizing apparatus according to an embodiment of the presentinvention;

FIG. 16 is an example of the labelling process of the patternrecognizing apparatus according to an embodiment of the presentinvention;

FIGS. 17A through 17D show the representations of compressing thelabelling process of the pattern recognizing apparatus according to anembodiment of the present invention;

FIG. 18 shows an example of the text extracting process of the patternrecognizing apparatus according to an embodiment of the presentinvention;

FIGS. 19A through 19D show examples of the partial area in the textextracting process of the pattern recognizing apparatus according to anembodiment of the present invention;

FIG. 20 shows the contiguous projecting method in the ruled lineextracting process of the pattern recognizing apparatus according to anembodiment of the present invention;

FIG. 21 shows the pattern projection result in the ruled line extractingprocess of the pattern recognizing apparatus according to an embodimentof the present invention;

FIG. 22 is a flowchart showing the ruled line extracting process of thepattern recognizing apparatus according to an embodiment of the presentinvention;

FIG. 23 shows the ruled line extracting process of the patternrecognizing apparatus according to an embodiment of the presentinvention;

FIG. 24 shows the method of completing an obscure ruled line in theruled line extracting process of the pattern recognizing apparatusaccording to an embodiment of the present invention;

FIG. 25 is a flowchart showing the method of completing an obscure ruledline of the pattern recognizing apparatus according to an embodiment ofthe present invention;

FIG. 26 shows the searching direction when an obscure ruled line iscompleted by the pattern recognizing apparatus according to anembodiment of the present invention;

FIG. 27 is a flowchart showing the one-character box extracting processof the pattern recognizing apparatus according to an embodiment of thepresent invention;

FIG. 28 is a flowchart showing the block character box extractingprocess of the pattern recognizing apparatus according to an embodimentof the present invention;

FIGS. 29A through 29E show the types of boxes and tables used in thepattern recognizing apparatus according to an embodiment of the presentinvention;

FIG. 30 is a flowchart showing the image reducing process of the patternrecognizing apparatus according to an embodiment of the presentinvention;

FIGS. 31A through 31E show the box-touching state determining process ofthe pattern recognizing apparatus according to an embodiment of thepresent invention;

FIG. 32 is a flowchart showing the box-touching state determiningprocess of the pattern recognizing apparatus according to an embodimentof the present invention;

FIGS. 33A through 33E show the types of deletion lines used in thepattern recognizing apparatus according to an embodiment of the presentinvention;

FIGS. 34A through 34C show the method of computing the feature of acorrected character used in the pattern recognizing apparatus accordingto an embodiment of the present invention;

FIG. 35 is a block diagram showing an example of the configuration ofthe basic character recognizing unit shown in FIG. 7;

FIG. 36 shows an example of the method of computing a feature vector inthe basic character recognizing unit shown in FIG. 7;

FIG. 37 shows an example of the method of computing the distance betweenthe feature vectors in the basic character recognizing unit shown inFIG. 7;

FIGS. 38A through 38C show the method of extracting a character segmentby the detailed identifying method for use with the basic characterrecognizing unit shown in FIG. 7;

FIG. 39 shows the method of detecting an end point in the detailedidentifying method for use with the basic character recognizing unitshown in FIG. 7;

FIG. 40 shows the method of detecting a change in angle in the detailedidentifying method for use with the basic character recognizing unitshown in FIG. 7;

FIGS. 41A and 41B show the correspondence among character segments inthe detailed identifying method for use with the basic characterrecognizing unit shown in FIG. 7;

FIG. 42 is a flowchart showing the process of the detailed identifyingmethod for use with the basic character recognizing unit shown in FIG.7;

FIGS. 43A through 43C show the method of completing a character by thebox-touching character recognizing unit shown in FIG. 7;

FIGS. 44A through 44D show the re-completing method by the box-touchingcharacter recognizing unit shown in FIG. 7;

FIGS. 45A through 45C show examples of a completed misread character bythe box-touching character recognizing unit FIG. 7;

FIG. 46 is a block diagram showing an example of the method of learninga character by the box-touching character recognizing unit FIG. 7;

FIG. 47 shows the method of generating a box-touching character by thebox-touching character recognizing unit shown in FIG. 7;

FIG. 48 shows an example of generating a box-touching character by thebox-touching character recognizing unit shown in FIG. 7;

FIG. 49 shows an example of a knowledge table for use in thebox-touching character recognizing unit shown in FIG. 7;

FIG. 50 shows an example of the type and amount of a change entered onthe knowledge table for use in the box-touching character recognizingunit shown in FIG. 7;

FIGS. 51A and 51B show examples of the re-recognized area emphasized bythe box-touching character recognizing unit shown in FIG. 7;

FIGS. 52A through 52D show the re-recognizing method using an emphasizedarea by the box-touching character recognizing unit shown in FIG. 7;

FIG. 53 is a flowchart showing the re-recognizing process using anemphasized area by the box-touching character recognizing unit shown inFIG. 7;

FIG. 54 is a block diagram showing an example of the characterre-recognizing method for use with the box-touching characterrecognizing unit shown in FIG. 7;

FIG. 55 is a block diagram showing the character re-recognizing processfor use with the box-touching character recognizing unit shown in FIG.7;

FIG. 56 shows the graphic meaning of a parameter in the statisticprocess performed by the character string recognizing unit shown in FIG.7;

FIG. 57 is a flowchart showing the statistic process performed by thecharacter string recognizing unit shown in FIG. 7;

FIG. 58 shows the graphic meaning of a parameter in the delimitingcharacter process performed by the character string recognizing unitshown in FIG. 7;

FIG. 59 is a flowchart showing the delimiting character processperformed by the character string recognizing unit shown in FIG. 7;

FIG. 60 shows the graphic meaning of a parameter in thesuperscript-stroke process performed by the character string recognizingunit shown in FIG. 7;

FIG. 61 is a flowchart showing the superscript-stroke process performedby the character string recognizing unit shown in FIG. 7;

FIG. 62 is a flowchart showing the process of computing thecharacter-detection-possibility data of the character string recognizingunit shown in FIG. 7;

FIG. 63 shows the method of quantizing the character detectionreliability of the character string recognizing unit shown in FIG. 7;

FIG. 64 shows the method of generating the frequency distribution of thecharacter string recognizing unit shown in FIG. 7;

FIG. 65 is a flowchart showing the method of computing the characterdetection reliability of the character string recognizing unit shown inFIG. 7;

FIG. 66 shows an example of a histogram distribution about the successand failure in character detection by the character string recognizingunit shown in FIG. 7;

FIG. 67 shows the method of computing the overlapping area of thesuccess and failure in character detection by the character stringrecognizing unit shown in FIG. 7;

FIG. 68 shows the flow of the process of detecting a character by thecharacter string recognizing unit shown in FIG. 7;

FIG. 69 shows the flow of the process of detecting a character in anon-statistic process performed by the character string recognizing unitshown in FIG. 7;

FIG. 70 is a block diagram showing an example of the configuration ofthe obscure character recognizing unit shown in FIG. 7;

FIG. 71 shows an example of the process performed by the deletion linerecognizing unit shown in FIG. 7;

FIG. 72 shows the flow of the clustering process performed by the uniquecharacter analyzing unit shown in FIG. 7;

FIG. 73 is a flowchart showing the clustering process performed by theunique character analyzing unit shown in FIG. 7;

FIG. 74 shows the flow of the character category determination resultcorrecting process performed by the unique character analyzing unitshown in FIG. 7;

FIG. 75 is a flowchart showing the character category determinationresult correcting process performed by the unique character analyzingunit shown in FIG. 7;

FIG. 76 shows an example of a list to be processed by the patternrecognizing apparatus according to the present invention;

FIG. 77 shows an example of the intermediate process result table foruse in the pattern recognizing apparatus according to an embodiment ofthe present invention;

FIG. 78 shows an example of the process order table for use in thepattern recognizing apparatus according to an embodiment of the presentinvention;

FIG. 79 shows an example of the intermediate process result table foruse in the pattern recognizing apparatus according to an embodiment ofthe present invention;

FIG. 80 shows an example of the intermediate process result table foruse in the pattern recognizing apparatus according to an embodiment ofthe present invention;

FIG. 81 shows an example of the intermediate process result table foruse in the pattern recognizing apparatus according to an embodiment ofthe present invention;

FIG. 82 shows an example of the intermediate process result table foruse in the pattern recognizing apparatus according to an embodiment ofthe present invention;

FIG. 83 is a block diagram showing the function of the patternrecognizing apparatus according to the third embodiment of the presentinvention;

FIG. 84 is a block diagram showing the function of the patternrecognizing apparatus according to the fourth embodiment of the presentinvention;

FIG. 85 is a block diagram showing the function of the patternrecognizing apparatus according to the fifth embodiment of the presentinvention;

FIG. 86 is a block diagram showing the function of the patternrecognizing apparatus according to the sixth embodiment of the presentinvention;

FIG. 87 is a block diagram showing the function of the patternrecognizing apparatus according to the seventh embodiment of the presentinvention;

FIG. 88 is a flowchart showing the operations performed by the patternrecognizing apparatus according to the eighth embodiment of the presentinvention;

FIG. 89 is a flowchart showing the operations performed by the patternrecognizing apparatus according to the ninth embodiment of the presentinvention;

FIG. 90 is a flowchart showing the operations performed by the patternrecognizing apparatus according to the tenth embodiment of the presentinvention;

FIG. 91 is a flowchart showing the operations performed by the patternrecognizing apparatus according to the eleventh embodiment of thepresent invention;

FIG. 92 is a block diagram showing the function of the patternrecognizing apparatus according to the twelfth embodiment of the presentinvention;

FIG. 93 shows the amount of the complexity of the pattern recognizingapparatus according to an embodiment of the present invention;

FIG. 94 is a flowchart showing the operations of the pattern recognizingapparatus shown in FIG. 92; and

FIG. 95 is a flowchart showing the operations of the pattern recognizingapparatus shown in FIG. 92.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

The pattern recognizing apparatus according to the first embodiment ofthe present invention is described by referring to the attacheddrawings.

FIG. 4 is a block diagram showing the functions of the patternrecognizing apparatus according to the first embodiment of the presentinvention.

In FIG. 4, a feature extracting unit 1000 extracts the firstpredetermined feature from an input pattern. A pattern recognizing unit1001 recognizes an input pattern by extracting the second predeterminedfeature from the input pattern from which the feature extracting unit1000 has extracted the first feature.

Thus, the pattern recognizing unit 1001 can recognize only an inputpattern having the first predetermined feature, and can perform arecognizing process on each environment of the input pattern, therebyimproving the precision of the recognizing process.

The pattern recognizing unit 1001 can also recognize an input pattern byextracting the second predetermined feature from the input pattern fromwhich the feature extracting unit 1000 has not extracted the firstfeature.

The pattern recognizing unit 1001 can remove the input pattern havingthe first predetermined feature from a process object, and only anappropriate process object can be selected and recognized, therebyimproving the precision of the recognizing process.

The pattern recognizing apparatus according to the second embodiment ofthe present invention is described by referring to the attacheddrawings.

FIG. 5 is a block diagram showing the functions of the patternrecognizing apparatus according to the second embodiment of the presentinvention.

In FIG. 5, a environment recognizing unit 1 extracts the first throughN-th states from an input image. A state extracted from the input imagerefers to, for example, the format such as a one-character box, afree-pitch box, table, etc. in which characters are written; therelationship between a character and its box; the obscurity of acharacter; the deformation of a character; the deletion of a characterusing a deletion line, etc.

A first pattern recognizing unit 2 exclusively recognizes a pattern of aprocess object in the first state. A second pattern recognizing unit 4exclusively recognizes a pattern of a process object in the secondstate. An N-th pattern recognizing unit 6 exclusively recognizes apattern of a process object in the N-th state.

The first through N-th pattern recognizing units 2, 4, and 6respectively comprise reliability computing units 3, 5, and 7, andcompute the reliability of the recognition results obtained by the firstthrough N-th pattern recognizing units 2, 4, and 6.

The environment recognizing unit 1 calls any of the first through N-thpattern recognizing units 2, 4, and 6 corresponding to the first throughN-th state to perform a recognizing process.

For example, when the environment recognizing unit 1 extracts the firststate from the input image, the pattern recognizing process to beperformed by the first pattern recognizing unit 2 is called for theprocess object in the first state. When the second state is extractedfrom the input image, the pattern recognizing process to be performed bythe second pattern recognizing unit 4 is called for the process objectin the second state. When the N-th state is extracted from the inputimage, the pattern recognizing process to be performed by the N-thpattern recognizing unit 6 is called for the process object in the N-thstate.

When the environment recognizing unit 1 extracts, for example, the firstand second states for a single process object, the pattern recognizingprocesses to be respectively performed by the first pattern recognizingunit 2 and second pattern recognizing unit 4 are called for the processobject.

Assume that the first state refers to the state in which characters arewritten in one-character boxes, the second state refers to the state inwhich characters are written in free-pitch character boxes, the thirdstate refers to the state in which a character touches its characterbox, the fourth state refers to the state in which a character isobscure, the fifth state refers to the state in which a character isdeformed, and the sixth state refers to the state in which a characteris corrected using deletion lines. With the above described statesdefined above, the first pattern recognizing unit 2 recognizes acharacter written in a one-character box, the second pattern recognizingunit 4 recognizes a character string written in a free-pitch box, thethird pattern recognizing unit recognizes a box-touching character, thefourth pattern recognizing unit recognizes an obscure character, thefifth pattern recognizing unit recognizes a deformed character, and thesixth pattern recognizing unit recognizes a corrected character.

When the environment recognizing unit 1 extracts a one-character boxfrom the input image, the first pattern recognizing unit 2 performs therecognizing process on the character written in the one-character box.When the environment recognizing unit 1 extracts a free-pitch box fromthe input image, the second pattern recognizing unit 4 performs therecognizing process on the character written in the free-pitch box. Whenthe environment recognizing unit 1 extracts a box-touching characterfrom the input image, the third pattern recognizing unit performs therecognizing process on the box-touching character. When the environmentrecognizing unit 1 extracts an obscure character from the input image,the fourth pattern recognizing unit performs the recognizing process onthe obscure character. When the environment recognizing unit 1 extractsa deformed character from the input image, the fifth pattern recognizingunit performs the recognizing process on the deformed character. Whenthe environment recognizing unit 1 extracts a candidate for a correctedcharacter from the input image, the sixth pattern recognizing unitperforms the recognizing process on the candidate for the correctedcharacter.

For example, when the environment recognizing unit 1 extracts abox-touching character touching a free-pitch box from the input image,the pattern recognizing unit 2 and the pattern recognizing unit 3perform the recognizing processes on the box-touching character touchingthe free-pitch box. When the environment recognizing unit 1 extracts abox-touching character with deletion lines which touches a free-pitchbox from the input image, the second pattern recognizing unit 4, thethird pattern recognizing unit, and the sixth pattern recognizing unitperform the recognizing processes on the box-touching character withdeletion lines which touches the free-pitch box.

When a plurality of states are extracted from a single process objectand a plurality of pattern recognizing units 2, 4, and 6 are called, theorder of the recognizing processes to be performed by the plurality ofpattern recognizing units 2, 4, and 6 is determined according to theprocess order table storing the order of calling the plurality ofpattern recognizing units 2, 4, and 6. Thus, the recognizing processesto be performed by the plurality of pattern recognizing units 2, 4, and6 are sequentially performed in the calling order until the reliabilitylarger than a predetermined threshold can be obtained by the reliabilitycomputing units 3, 5, and 7 in the recognizing processes performed bythe pattern recognizing units 2, 4, and 6.

For example, when the environment recognizing unit 1 extracts abox-touching character which touches a free-pitch box from an inputimage, the third pattern recognizing unit first performs the recognizingprocess and then the second pattern recognizing unit performs therecognizing process on the box-touching character which touches thefree-pitch box. When the environment recognizing unit 1 extracts abox-touching character with deletion lines which touches a free-pitchbox from an input image, the third pattern recognizing unit firstperforms the recognizing process, the sixth pattern recognizing unitperforms the recognizing process, and then the second patternrecognizing unit 4 performs the recognizing process on the box-touchingcharacter with deletion lines which touches the free-pitch box.

FIG. 6 is a block diagram showing the configuration of an embodiment ofthe environment recognizing unit 1 shown in FIG. 1.

In FIG. 6, a state extracting unit 1 a extracts the first through theN-th states from an input image.

A recognizing process controlling unit 1 b calls one or a plurality ofthe first through N-th pattern recognizing units 2, 4, and 6 shown inFIG. 5 corresponding to the first through N-th states extracted by thestate extracting unit 1 a for use in a recognizing process.

A process order table 1 f stores the process order indicating theprocess order of the first through N-th pattern recognizing units 2, 4,and 6 when the plurality of recognizing units are called from among thefirst through N-th pattern recognizing units 2, 4, and 6.

A process order control rule storage unit 1 d stores the callingprocedure indicating the recognizing unit to be called from among thefirst through N-th pattern recognizing units 2, 4, and 6 based on thefirst through N-th states extracted by the state extracting unit 1 a.

An intermediate process result table generating unit 1 c generates anintermediate process result table indicating the process order of thefirst through N-th pattern recognizing units 2, 4, and 6 according tothe calling procedure stored in the process order control rule storageunit 1 d and the process order stored in the process order table 1 f.

A process performing rule storage unit 1 e stores the procedureindicating the next process to be performed based on the result of therecognizing process entered in the intermediate process result table.

As described above, the pattern recognizing apparatus shown in FIG. 5extracts the state of a process object from an input image, selects anappropriate recognizing process for the state for each process object sothat an appropriate pattern recognizing process can be performed foreach state on an input image having various states, thereby performingthe recognizing process with high precision. Since the process object isevaluated when the state is extracted and also when a recognizingprocess is performed on the process object, the precision of therecognizing process can be furthermore improved.

For example, extracting the state of a process object from an inputimage, performing the pattern recognizing process exclusively for thefirst state on the process object having the first state, and performingthe pattern recognizing process exclusively for the second state on theprocess object having the second state suppress the mutual influencebetween the recognizing process on the process object having the firststate and the recognizing process on the process object having thesecond state, thereby realizing a recognizing process with highprecision.

Furthermore, performing a plurality of recognizing processes on a singleprocess object until the reliability of the recognizing process reachesa predetermined value improves the reliability of the recognizingprocess with enhanced precision.

FIG. 7 is a block diagram showing a practical configuration of thepattern recognizing apparatus according to an embodiment of the presentinvention.

In FIG. 7, an environment recognizing system 11 extracts the state of aninput image, and calls one or a plurality of a basic characterrecognizing unit 17, a character string recognizing unit 15, abox-touching character recognizing unit 13, an obscure characterrecognizing unit 19, a deformed character recognizing unit 21 of acharacter recognizing unit 12 and a deletion line recognizing unit 26and noise recognizing unit 28 of a non-character recognizing unit 25based on the extracted state.

The character recognizing unit 12 performs a character recognizingprocess for each state of an input image and comprises the basiccharacter recognizing unit 17 for recognizing a character, the characterstring recognizing unit 15 for performing a character recognizingprocess B and a character detecting process B on a character string, thebox-touching character recognizing unit 13 for performing a characterrecognizing process A and a character detecting process A on abox-touching character, the obscure character recognizing unit 19 forperforming a character recognizing process C and a character detectingprocess C on an obscure character, the deformed character recognizingunit 21 for performing a character recognizing process D and a characterdetecting process D on a deformed character, and the unique characterrecognizing unit 23 for performing a unique character recognizingprocess E and a unique character detecting process E on a uniquecharacter.

The basic character recognizing unit 17, character string recognizingunit 15, box-touching character recognizing unit 13, obscure characterrecognizing unit 19, deformed character recognizing unit 21, and uniquecharacter recognizing unit 23 respectively comprise knowledge tables 14,16, 18, 20, 22 and 24 storing the knowledge of the character recognizingmethod. The knowledge table 14 stores the knowledge of a box-touchingstate and reliability of a recognizing process and the knowledge of theoverlapping portion pattern method. The knowledge table 16 stores, forexample, the knowledge of the reliability of a detecting process and theknowledge of the method of combining a detecting process and arecognizing process. The knowledge table 18 stores, for example, theknowledge about a detail recognizing method.

With the above described configuration, a recognizing process can beperformed on an input image having various states by selecting andreferring to appropriate knowledge for each state, thereby improving theprecision of the recognizing process.

Furthermore, a recognizing process can be performed on an input imagehaving various states by selecting and referring to appropriaterecognizing dictionary for each state, thereby improving the precisionof the recognizing process.

Additionally, a recognizing process can be performed on an input imagehaving various states by selecting appropriate identification functionfor each state, thereby improving the precision of the recognizingprocess.

The non-character recognizing unit 25 performs a non-characterrecognizing process for each state of an input image, and comprises thedeletion line recognizing unit 26 for performing a non-characterrecognizing process F and a non-character detecting process F on acharacter with a deletion line, and the noise recognizing unit 28 forperforming a non-character recognizing process G and a non-characterdetecting process G on a noise.

The deletion line recognizing unit 26 and noise recognizing unit 28respectively comprise knowledge tables 27 and 29 storing the knowledgeof non-character recognizing methods.

Thus, the recognizing process can be performed with high precision withless characters mistaken for non-characters and with less non-charactersmistaken for characters by performing the recognizing process oncharacters and non-characters separately.

FIG. 8 is a flowchart showing an example of the entire process of theenvironment recognizing system 11.

In FIG. 8, an input image is pre-processed in step S1. The pre-processof the input image is performed by labelling the input image binarizedby a facsimile, a scanner, etc., and the input image and the labelledimage are stored. The input image and the labelled image are stored in away that they can be accessed at any time in the subsequent processes.

FIG. 9 is a flowchart showing the pre-process of the input image shownin FIG. 8.

In FIG. 9, a binarized input image is labelled in step S11 so that alink pattern can be extracted and labelled, and the extracted labelledimage and the input image are stored. At this time, the memory capacitycan be saved by compressing the labelled link pattern by adding andsubtracting the circumscribing rectangle. By compressing the labelledlink pattern, a document/form of the size A4 (about 3000×4000 pixel)entered by a scanner at the high resolution of, for example, 400 dpi canbe represented within several hundred kilobytes.

Next, a layout is analyzed in step S2 shown in FIG. 8. The analysis ofthe layout is performed by recognizing a text, extracting a ruled line,extracting a character box, determining the type of box and table,determining the existence of a box-touching character, and recognizing adrawing based on the size, the arrangement, etc. of a labelled linkpattern.

FIG. 10 is a flowchart showing the layout analysis shown in FIG. 8.

In FIG. 10, the text is recognized in step S21. In this textrecognition, the size of the labelled link pattern is analyzed, and alink pattern of a relatively small size is extracted and defined as acandidate for a character. Then, a text is extracted by integrating thecandidate with the adjacent character candidate.

Then, a ruled line is extracted in step S22. The ruled line can beextracted by searching a pattern indicating a larger histogram value inthe vertical or horizontal direction for the link patterns which are notrecognized as text in step S21.

Then, a character box is extracted in step S23. The character box isextracted by detecting ruled lines corresponding to the four sides ofthe box from the ruled lines extracted in step S22.

Then, in step S24, the type of box and table is discriminated. Bydiscriminating the type of box and table, the type of the character boxextracted in step S23 is discriminated and the attribute of the type ofthe character box is assigned. The attribute of the type of thecharacter box can be a one-character box, a block character box, afree-pitch character box, a table, etc.

In step S25, it is determined whether or not a box-touching characterexists. The determination of the box-touching character is performed bydetecting a crossing pattern when the character box is searched alongthe character box line. If a crossing pattern exists, it is determinedthat the character touches the character box. Another character in theadjacent character box may be partly out of its box and enters thepresent box. Therefore, in such cases, the character partly out of itsown box and entering the present box is not defined as a box-touchingcharacter to the present character box.

Then, a drawing is recognized in step S26. In the recognition of thedrawing, a link pattern of a relatively large size which has not beenassigned the attribute such as a text, character box, table, etc. isassigned an attribute of the drawing.

A quality is analyzed in step S3 shown in FIG. 8. In the qualityanalysis, obscurity, deformation, etc. is detected if any in the inputimage. The analysis can be a global quality analysis and a local qualityanalysis.

In this quality analysis, it is determined that an obscure character isdetected in a predetermined area if the value obtained by dividing (thenumber of link areas having the sizes of the area, length, and widthsmaller than the respectively predetermined threshold values) by (thenumber of all link areas in the predetermined area) is larger than apredetermined value.

It is also determined that an obscurity is detected in a predeterminedarea if the value obtained by dividing (the total length of thecompleted portion when the obscure ruled line is completed) by (thetotal length of each ruled line) is larger than a predetermined valueaccording to the information into which an obscure ruled line inextracting the ruled line is partly integrated.

Furthermore, it is determined that deformation is detected in apredetermined area if the value obtained by dividing (the number of linkareas indicating the black picture element density larger than apredetermined threshold) by (the total number of link areas in thepredetermined area) is larger than a predetermined value.

FIG. 11 is a flowchart of the quality analysis shown in FIG. 8.

In FIG. 11, a global quality analysis is performed in step S31. Theglobal quality analysis is performed on the entire document/form, anddetermines whether or not the threshold used in binarizing the inputimage is appropriate, whether or not the quality of the document/formwith noises given to them after transmitted through facsimile isacceptable, and whether or not obscurity or deformation has beengenerated.

Then, a local quality analysis is performed in step S32. The localquality analysis is performed by checking whether or not obscurity ordeformation has been generated or whether or not noises have beengenerated in each of the areas assigned attributes of a one-characterbox, a text, a free-pitch character box, a table, etc. in the layoutanalysis.

Next, a correction analysis is performed in step S4 shown in FIG. 8. Inthe correction analysis, a deletion line is extracted from an inputimage, and a character recognizing process can be omitted for thecharacter corrected with the deletion line.

FIG. 12 is a flowchart showing the correction analysis shown in FIG. 8.

In FIG. 12, a correction feature is extracted in step S41. In thiscorrection feature extraction, a feature of a corrected character isextracted. The corrected character can be one of the four types ofcharacter, that is, a deformed character, a character removed withdouble lines, a character removed with a diagonal line, and a characterremoved with a symbol ‘x’. The feature of each of the correctedcharacters can be extracted by computing the black picture elementdensity, line density, Euler number, histogram value, etc.

Then, a correction character candidate extracting process is performedin step S42. A candidate for a corrected character is extracted based onthe difference between the corrected character and the non-correctednormal character in a feature space representing the features of thecorrected character.

Next, a character/non-character recognizing control is performed in stepS5 shown in FIG. 8. In the character/non-character recognizing control,it is determined which should be called from among the basic characterrecognizing unit 17, character string recognizing unit 15, box-touchingcharacter recognizing unit 13, obscure character recognizing unit 19,deformed character recognizing unit 21 of the character recognizing unit12 and the deletion line recognizing unit 26 and noise recognizing unit28 of the non-character recognizing unit 25, based on the state of aninput image extracted in steps S2 through S4 shown in FIG. 8. In thiscontrol, the intermediate process result table is read, process ordercontrol rule is followed and terminated, and the process is performedunder process execution rules.

The process order control rule shows the procedure of calling whichshould be called from among the basic character recognizing unit 17,character string recognizing unit 15, box-touching character recognizingunit 13, obscure character recognizing unit 19, and deformed characterrecognizing unit 21 of the character recognizing unit 12 and thedeletion line recognizing unit 26 and noise recognizing unit 28 of thenon-character recognizing unit 25, based on the state extracted by theenvironment recognizing system 11.

The process execution rule indicates the procedure of the process to beperformed next, based on the result of the recognizing process calledaccording to the process order control rule.

The intermediate process result table includes the state of the inputimage extracted in steps S2 through S4 shown in FIG. 8 for each of theareas assigned the attributes of a one-character box, text, free-pitchcharacter box, table, etc. through the layout analysis. On theintermediate process result table, the processes called according to theinput process order control rule are entered in the process order storedin the process order table.

For example, when the environment recognizing system 11 extracts acharacter, the basic character recognizing unit 17 is called to performa recognizing process on the character. When the environment recognizingsystem 11 extracts a text in step S21 shown in FIG. 10, the characterstring recognizing unit 15 is called to perform a recognizing process onthe text. When the environment recognizing system 11 extracts abox-touching character in step S25 shown in FIG. 10, the box-touchingcharacter recognizing unit 13 is called to perform a recognizing processon the box-touching character. When the environment recognizing system11 determines in step S32 that the value obtained by dividing (thenumber of link areas having the sizes of the area, length, and widthsmaller than the respectively predetermined threshold values) by (thenumber of all link areas in the predetermined area) is larger than apredetermined value, the obscure character recognizing unit 19 is calledto perform a recognizing process on the character in this area. When theenvironment recognizing system 11 determines in step S32 that the valueobtained by dividing (the number of link areas indicating the blackpicture element density larger than a predetermined threshold) by (thetotal number of link areas in the predetermined area) is larger than apredetermined value, the deformed character recognizing unit 21 iscalled to perform a recognizing process on the character in this area.When the environment recognizing system 11 extracts a candidate for acorrected character in step S42 shown in FIG. 12, the deletion linerecognizing unit 26 is called to perform a recognizing process on thecandidate for a corrected character. When the environment recognizingsystem 11 detects a noise in step S32 shown in FIG. 11, the noiserecognizing unit 28 is called to perform a recognizing process on thenoise.

FIG. 13 is a flowchart showing the control of character recognizingprocess/non-character recognizing process shown in FIG. 8.

In FIG. 13, the intermediate process result table is read and theprocess order control rules are executed in step S51.

Then, it is determined in step S52 whether or not the process has beencompleted. It is determined that the process has been completed when allprocesses on the intermediate process result table have been completedbased on the process order control rules, and all process instructioncolumns on the intermediate process result table contain completionentries. If it is determined that the process has not been completedyet, then control is passed to step S53, returned to step S51 afterperforming the process according to the process execution rules, and theabove described processes are repeated until it is determined in stepS52 that the process has been completed.

FIG. 14 is a block diagram showing the configuration of the patternrecognizing apparatus according to an embodiment of the presentinvention.

In FIG. 14, an image storage unit 41 stores a form image. A processcondition storage unit 42 stores definitions such as the layoutstructure of the form and read character information, for example, theposition, type, and size of a character box, type of characters, numberof characters, etc. A labelled image storage unit 43 stores compressedand labelled images.

An environment recognizing system 30 comprises a layout analyzing unit31 and correction analyzing unit 32. An environment recognizing system38 comprises a unique character analyzing unit 39 and a completiondetermining unit 40. A character recognizing system/non-characterrecognizing system 33 comprises a basic character recognizing unit 34, ablack-character-box touching character recognizing unit 35, a free-pitchcharacter string recognizing unit 36, and a deletion line recognizingunit 37.

The layout analyzing unit 31 refers to the definitions stored in theprocess condition storage unit 42 for a labelled image stored in thelabelled image storage unit 43, and extracts ruled lines, characterboxes, and black-character-box touching character. The method ofpreliminarily storing format information about the position and size ofa character box and information about the pose of a character box aslist data, and extracting ruled lines and character boxes according tothe list data is disclosed by, for example, Tokukaisho 62-21288 andTokukaihei 3-126186.

As described in, for example, Tokukaihei 6-309498 and Tokukaihei7-28937, ruled lines and character boxes can be extracted withoutentering format information such as the position and size of a characterbox.

The correction analyzing unit 32 extracts a candidate for a deletionline. The unique character analyzing unit 39 analyzes a unique characterhaving a personal handwritten feature. The completion determining unit40 determines the completion of a character recognizing process, andoutputs a character recognition result when it is determined that theprocess has been completed.

The basic character recognizing unit 34 recognizes characters detectedone by one. The black-character-box touching character recognizing unit35 removes a character box from a black-character-box touchingcharacter, completes an obscure character by removing the character box,and then recognizes the character. The free-pitch character stringrecognizing unit 36 recognizes a character in a character string inconsideration of the detection reliability when the character isdetected from the character string. The deletion line recognizing unit37 recognizes a deletion line based on the black picture element densityof a corrected character, line density, Euler number, histogram, etc.

An intermediate process result table 44 stores the process orderindicating which process is to be performed, the character recognizingsystem or the non-character recognizing system 33, and the result of theprocess, based on the state extracted by the environment recognizingsystems 30 and 38.

FIG. 15 is a block diagram showing a practical configuration of thecharacter recognizing system to which the pattern recognizing apparatusshown in FIGS. 5 through 7 is applied.

In FIG. 15, a central processing unit (CPU) 51 performs variousprocesses. A program memory 50 stores a program to be executed by theCPU 51. A image memory 52 stores image data in a bit map format. A workmemory 53 is used in processing an image. A scanner 59 optically readsan image. A memory 58 temporarily stores information read by the scanner59. A dictionary file 60 stores the feature of each character image. Adisplay 56 displays a recognition result. A printer 57 prints arecognition result. An interface circuit 55 functions for the display 56and printer 57. A bus 54 connects the CPU 51, program memory 50, imagememory 52, work memory 53, memory 58, dictionary file 60, and interfacecircuit 55.

The character recognizing system temporarily stores image data read bythe scanner 59 in the memory 58, and develops the image data in the bitmap format into the image memory 52. Then, a pattern extracting processis performed on the binary image data copied from the image memory 52 tothe work memory 53. Based on the result, a character image is detectedfrom the image data read by the scanner 59. The feature of the detectedcharacter image is compared with the feature data stored in thedictionary file 60 to recognize a character. Then, the recognitionresult is output to the display 56 or printer 57.

With this character recognizing system, the pattern recognizingapparatus shown in FIGS. 5 through 7 can be realized as having thefunction of the CPU 51 for performing the process according to theprogram stored by the program memory 50.

The configurations of the environment recognizing system 11, characterrecognizing unit 12, and non-character recognizing unit 25 shown in FIG.7 are practically described below.

FIG. 16 shows the labelling process performed in step S11 shown in FIG.9.

When a binary image comprising ‘0’ and ‘1’ is input to a labelling unit70 as shown in FIG. 16, the labelling unit 70 extracts a link patterncomprising link picture elements from the input binary image, generatesa labelled image assigned a label for each link pattern, and stores theimage in a labelled image storage unit 71. For example, when a binaryimage 72 comprising ‘0’ and ‘1’ is input, a labelled image 73 isgenerated with labels ‘1’, ‘2’, and ‘3’ assigned to each link pattern.

If a single image contains 255 link patterns, a total of 255 labels arerequired. Since one picture element requires 8 bits, the storagecapacity of the labelled image storage unit 71 equals 8 times of thenumber of picture elements of the entire image, thereby requiring alarge storage capacity to store the labelled image.

FIG. 17 shows the method of reducing the storage capacity required forthe labelled image storage unit 71 by compressing the labelled image 73shown in FIG. 16.

In FIG. 17, labels ‘1’ and ‘2’ are respectively assigned to linkpatterns A₁ and A₂ shown in FIGS. 17A and 17B. Rectangle B₁circumscribes link pattern A₁ and rectangle B₂ circumscribes linkpattern A₂ as shown in FIG. 17C. Circumscribing rectangles B₁ and B₂ canbe specified by the coordinate (x₁, y₁) of the left-top vertex and thecoordinate (x₂, y₂) of the right-bottom vertex of circumscribingrectangles B₁ and B₂ as shown in FIG. 17D.

Then, it is determined whether or not rectangle B₁ circumscribing linkpattern A₁ overlaps rectangle B₂ circumscribing link pattern A₂. Ifrectangle B₁ circumscribing link pattern A₁ does not overlap rectangleB₂ circumscribing link pattern A₂, then the coordinates (x₁, y₁) of theleft-top vertexes and the coordinates (x₂, y₂) of the right-bottomvertex of circumscribing rectangles B₁ and B₂ respectively are stored.

On the other hand, if rectangle B₁ circumscribing link pattern A₁overlaps rectangle B₂ circumscribing link pattern A₂, thencircumscribing rectangles B₁ and B₂ are divided into smaller rectangularareas so that these circumscribing rectangles may not further overlapanother circumscribing rectangle. Then, it is determined to which doeseach of the divided rectangular areas belongs, the originalcircumscribing rectangle B₁ and B₂. Link patterns A₁ and A₂ arerepresented by arithmetic operations such as addition, subtraction, etc.

For example, as shown in FIG. 17C, link pattern A₁ can be represented bythe difference between rectangular area (1-1) and rectangular areas(1-2) as shown by the following equation where (1-1) indicates themaximum rectangular area belonging to link pattern A₁, and (1-2)indicates rectangular areas contained in rectangular area (1-1).

A ₁=(1-1)−(1-2)

Similarly, link pattern A₂ can be represented by the difference betweenrectangular area (2-1) and rectangular areas (2-2) plus rectangular area(2-3) as shown by the following equation where (2-1) indicates themaximum rectangular area belonging to link pattern A₂, (2-2) indicatesrectangular area contained in rectangular area (2-1), and (2-3)indicates rectangular areas contained in rectangular areas (2-2).

A ₂=(2-1)−(2-2)+(2-3)

Thus, the storage capacity required to store labelled images can besmaller by reducing the volume of information representing link patternsby representing the link pattern by a rectangle circumscribing a seriesof picture elements.

The method of compressing labelled images is disclosed by Tokukaihei8-55219.

FIG. 18 is a flowchart showing an embodiment of the text recognizingprocess in step S21 shown in FIG. 10.

As shown in FIG. 18, a document is read by a scanner and the image dataof the read document is stored in memory in step S61.

Then, in step S62, only a specified strip portion in the horizontaldirection is observed among the image data read in step S61, thelabelling process is performed on the observed portion, and acircumscribing rectangle of black link picture elements is obtained.

For example, if plural documents A, B, and C are processed as processobjects, the area of a character string 81 of document A as shown inFIG. 19A is within section A as shown in FIG. 19D, the area in characterstring 82 of document B shown in FIG. 19B is within section A as shownin FIG. 19D, and the area in character string 83 of document C shown inFIG. 19C is within section B as shown in FIG. 19D, then only theportions in sections A and B are observed and the labelling processesare performed in the strip portion only to obtain a rectanglecircumscribing of linked black picture elements.

Extracted next in step S63 is only a circumscribing rectangle indicatingthe difference, smaller than the threshold thy, between the heightobtained in step S62 and a predetermined height ylen, and indicating thedifference, smaller than the threshold thx, between the width obtainedin step S62 and the width xlen preliminarily obtained. Then, thecoordinate of the circumscribing rectangle in the y direction (verticaldirection) is obtained and stored in the memory.

Next, in step S64, a wide area having the coordinate as a center pointalong the y direction obtained in step S63 is observed with the width ofthe rectangle extracted in step S62 equaling the width of an image.

In step S65, a circumscribing rectangle of linked black picture elementsis obtained by labelling the wide area obtained in step S64.

Extracted next in step S66 is only a circumscribing rectangle indicatingthe difference, smaller than the threshold thy, between the heightobtained in step S65 and a predetermined height ylen, and indicating thedifference, smaller than the threshold thx, between the width obtainedin step S65 and the width xlen preliminarily obtained. The extractedcircumscribing rectangle is stored in the memory.

Next, in step S67, rectangles extracted in step S66 are sorted based onthe x coordinate. The pitch is computed from the intervals of the centerlines of the extracted rectangle. When a predetermined number th or moreof rectangles indicating the difference between the computed pitch and apreliminarily obtained pitch smaller than the threshold thpitch arearranged in the horizontal direction, they are output as text.

This text extracting method is described by, for example, Tokukaihei8-171609.

Described below is an embodiment of the ruled line extracting process instep S22 shown in FIG. 10.

In the ruled line extracting process, the linked pattern obtained in thelabelling process is divided into plural sections in the horizontal andvertical directions. The contiguous projection value of the linkedpattern is computed within each section of the pattern dividedhorizontally and vertically. Thus, ruled lines are extracted bydetecting a portion of a predetermined length of line through theapproximation of a rectangle.

A contiguous projection in this example refers to a sum of a projectionvalue of an object row or column and a projection value of a surroundingrow or column. The projection values of the object row or column areobtained by computing the total number of black picture elements in therow or column.

FIG. 20 shows the contiguous projection process.

In FIG. 20, the projection value in row i is p(i), and the contiguousprojection value P(i) can be computed by the following equation (1).

P(i)=p(i−j)+ . . . +p(i)+ . . . +p(i+j)  (1)

In the example shown in FIG. 20, j=1 in the equation (1).

FIG. 21 shows an example of the projection value of a partial pattern.

Assuming that the projection value Ph(i) in the horizontal direction jis HP(i) and the projection value Pv(j) in the vertical direction i isVP(j) in the rectangle 84 having the length L_(y) and the width L_(x) inFIG. 21, HP(1)=HP(n)=m, HP(2)˜HP(n−1)=2, VP(1)=VP(m)=n, VP(2)˜VP(m−1)=2.

If straight lines forming part of the rectangle 84 exist, the projectionvalue is large. Therefore, the straight lines forming part of ruledlines can be extracted.

For example, a candidate for a straight line forming part of ruled linescan be extracted by detecting a partial pattern indicating the ratio,equal to or smaller than a predetermined threshold, of a contiguousprojection value to each of the lengths of vertical and horizontaldivisions.

FIG. 22 is a flowchart showing the ruled line extracting process.

In FIG. 22, it is determined in step S601 whether or not the ratio of acontiguous projection value to each of the lengths of vertical andhorizontal divisions is equal to or larger than a predeterminedthreshold. If it is not equal to or larger than the predeterminedthreshold, then control is passed to step S602, and it is assumed thatno lines forming part of ruled lines exist.

If it is determined in step S601 that the ratio of a contiguousprojection value to each of the lengths of vertical and horizontaldivisions is equal to or larger than a predetermined threshold, thencontrol is passed to step S603, and it is assumed that lines formingpart of the ruled lines exist.

It is determined in step S604 whether or not the pattern regarded as aline in step S603 touches a line above or below the pattern. If it isdetermined that the pattern does not touch a line above or below thepattern, control is passed to step S605, and the pattern is defined as aline forming part of a rectangle.

If it is determined in step S604 that the pattern regarded in step S603as a line touches the lines above and below the pattern, then control ispassed to step S606 and the pattern is integrated into the lines aboveand below the pattern. In step S607, the lines integrated in step S606are detected as rectangular lines. For example, the three rectangularlines 85 as shown by (A) in FIG. 23 are integrated, and a rectangularline 86 indicated by (B) in FIG. 23 is obtained. Then, ruled lines areextracted by searching for the rectangular lines obtained in step S605or S607.

The above described ruled line extracting process is described byTokukaihei 6-309498.

FIG. 24 shows the search method performed while completing obscure ruledlines in the ruled line extracting process in step S22 shown in FIG. 10.

The method of completing the obscure ruled lines is followed to searchfor a pattern forming a straight line. Even if an area without a patternexists in the searching direction, it is assumed that a pattern existsin a blank area containing the number of picture element smaller than apredetermined value.

For example, when a picture element 92 forming part of a straight line91 is retrieved from the straight line 91 as shown in FIG. 24, a blankarea 93 containing the number of picture elements smaller than apredetermined value is searched with the picture elements 92 assumed toexist.

FIG. 25 is a flowchart showing the method of completing an obscure ruledline in the ruled line extracting process.

In FIG. 25, the X coordinate of the thinnest portion of the pattern in apredetermined rectangular range is computed in step S71.

In step S72, a center point of a pattern at the X coordinate computed instep S71 is computed. In step S73, the center point of the patterncomputed in step S72 is set as a search start point. The search startpoint is set at the thinnest portion of the pattern because there is asmall possibility that the thinnest portion is a character, and thestraight line forming part of the character box can be more probablydetected.

In step S74, the search direction for a straight line is set to ‘right’.

Then, the initial value of the variable K is set to 0 to count thelength of the blank area in step S75.

In step S76, the start point obtained in step S73 is set as the currentposition of a pattern search.

In step S77, it is determined whether the current search position set instep S76 is in the range of the rectangle recognized in step S71. If thecurrent search position is not in the range of the rectangle observed instep S71, control is passed to stp S86.

If it is determined in step S77 that the current search position is inthe range of the rectangle observed in step S71, control is passed tostep S78, and it is determined whether or not a pattern is positionednext to the current search position in the search direction. A patternpositioned next to the current search position in the search directionrefers to a pattern 102 next to the right of a pattern 101 as shown inFIG. 26. If it is determined that the pattern 102 is positioned next tothe current search position in the search direction, then control ispassed to step S81, and the pattern 102 next to the current position inthe search direction is set as the current position.

If it is determined in step S78 that a pattern is not positioned next tothe current position in the search direction, control is passed to stepS79, and it is determined whether or not a pattern is positioneddiagonally above or below the current position in the search direction.

A pattern positioned diagonally above or below the current position inthe search direction refers to a pattern 104 a or a pattern 104 bdiagonally above or below a pattern 103 as shown in FIG. 26. If it isdetermined that the patterns 104 a and 104 b are positioned diagonallyabove or below the current position in the search direction, thencontrol is passed to step S83, and the patterns 104 a and 104 bdiagonally above or below the current position are defined as thecurrent position. If there are two patterns 104 a and 104 b positioneddiagonally above or below the current position in the search direction,then one of the patterns 104 a and 104 b is set as the current searchposition.

If it is determined in step S79 that the patterns 104 a and 104 b arenot positioned diagonally above or below the current position in thesearch direction, then control is passed to step S80, and it isdetermined whether or not the variable K for use in counting the lengthof a blank area is equal to or smaller than a predetermined threshold.If the variable K for use in counting the length of a blank area isequal to or smaller than a predetermined threshold, then control ispassed to step S84, and the picture element adjacent in the searchdirection and not forming part of the pattern is defined as the currentposition. For example, It is assumed that there is a pattern for a blankarea 93 having a predetermined number of or less picture elements asshown in FIG. 24, and a searching process is performed.

In step S85, the variable K for use in counting the length of a blankarea is increased by 1 dot, and control is returned to step S77.

If it is determined in step S80 that the variable K for use in countingthe length of a blank area is not equal to or smaller than apredetermined threshold, then control is passed to step S86, and it isdetermined whether or not the search direction is set to ‘right’. If itis not set to ‘right’, then the process terminates.

When the search direction is set to ‘right’ in step S86, then control ispassed to step S87 and the search direction is set to ‘left’. Then, theprocesses in steps S75 through S85 are similarly repeated as performedwhen the search direction is set to ‘right’.

When a process is performed with a search direction set to ‘left’, apattern positioned next to the current search position in the searchdirection refers to a pattern 106 next to the left of a pattern 105 asshown in FIG. 26. A pattern positioned diagonally above or below thecurrent position in the search direction refers to a pattern 108 a or apattern 108 b diagonally above or below a pattern 107 as shown in FIG.26.

Next, the character box extracting process in step S23 is describedbelow.

FIG. 27 is a flowchart showing an embodiment of the one-character boxextracting process.

As shown in FIG. 27, a searching process is performed on a patterndetected in the process shown in FIG. 22 as a line of a rectangle instep S91. At this time, a searching process is performed on a blank areaof a predetermined length assuming that a pattern exists as shown in theflowchart in FIG. 25, and an obscurity problem can be solved.

Next, it is determined in step S92 after a search in step S91 whether ornot a pattern is disconnected at a predetermined length. If it is notdisconnected at the predetermined length, then control is passed to theblock character box extracting process shown in FIG. 28. If the patternis disconnected at a predetermined length, then control is passed tostep S93, and a searched lines are combined into a straight line.

Next, as shown in step S94, straight lines forming a rectangle areextracted from the straight lines detected in step S93.

It is determined in step S95 whether or not the size of a portionencompassed by four straight lines is within a predetermined range for aone-character box in an image. If it is determined that the size of theportion encompassed by four straight lines is within the predeterminedrange for the one-character box in the image, then control is passed tostep S96 and the portion encompassed by the four straight lines isregarded as a one-character box. If it is determined that the size ofthe portion encompassed by four straight lines is not within thepredetermined range for the one-character box in the image, then controlis passed to step S97 and the portion encompassed by the four straightlines is not regarded as a one-character box.

FIG. 28 is a flowchart showing an embodiment of the block character boxextracting process.

As shown in FIG. 28, it is determined in step S101 whether or not ahorizontal line detected in a searching process is longer than apredetermined value. If the horizontal line detected in a searchingprocess is shorter than a predetermined value, then control is passed tostep S102, and the horizontal line is not regarded as a horizontal lineforming part of a character box. If it is determined that the horizontalline detected in a searching process is equal to or larger than apredetermined value, then control is passed to step S102, and thehorizontal line detected in the searching process is regarded as ahorizontal line forming part of the character box.

Two adjacent horizontal lines forming part of a character box areextracted from the horizontal lines extracted in step S103 as shown instep S104.

Then, a range encompassed by the two horizontal lines forming part ofthe character box extracted in step S104 is regarded as a blockcharacter box for one row in step S105.

In step S106, vertical lines are detected by extracting the linesforming a rectangle detected in the process shown in FIG. 22.

Then, in step S107, the vertical lines detected in step S106 issearched. It is determined in step S108 whether or not the verticallines have reached the horizontal lines which form part of the characterbox and are detected in step S104. If the vertical lines have notreached the horizontal lines forming part of the character box, thencontrol is passed to step S109, the vertical lines are removed from thecandidates for vertical lines. When the vertical lines touch the upperand lower sides of the character box, control is passed to S110 and thevertical lines are regarded as candidates for the vertical lines formingpart of the character box.

In step S111, it is determined whether a process object is a regulartable-form block character box or an irregular table-form blockcharacter box. If the process object is a regular table-form blockcharacter box, then control is passed to step S112, and the intervalbetween the vertical lines regarded as candidates for vertical linesforming part of a character lines is computed in step S110, and ahistogram indicating the relationship between the interval of thecomputed vertical lines and the frequency of the vertical lines iscomputed.

In step S113, the vertical lines making intervals different from otherintervals are removed from candidates for the vertical lines formingpart of the character box within a range encompassed by two adjacenthorizontal lines forming part of the character box. The remainingvertical lines are regarded as vertical lines forming part of thecharacter box, thereby terminating the process.

If it is determined in step S111 that the process object is an irregulartable-form block character box, then all candidates determined in stepS110 for the vertical lines are regarded as vertical lines forming partof the character box, thereby terminating the process.

Described below is the character-box type/table discriminating processin step S24 shown in FIG. 10.

FIG. 29 shows an example of the character box and table extracted in thecharacter box extracting process in step S23 shown in FIG. 10.

FIG. 29A shows a one-character box. FIG. 29B shows a free-pitchcharacter box. FIG. 29C shows a block character box. FIG. 29D shows aregular table. FIG. 29E shows an irregular table. The one-character boxis assigned an attribute of a one-character box. The free-pitchcharacter box is assigned an attribute of a free-pitch character box.The block character box is assigned an attribute of a block characterbox. The table is assigned an attribute of a table.

The character box extracting process and the character-box type/tablediscriminating process are described by Tokukaihei 7-28937.

Described next is the process of determining the existence of acharacter touching a character box in step S25 shown in FIG. 10. In thisprocess, an original input image is reduced using the reduction ratio of1/n in the OR process, and the process of determining the existence of abox-touching character is then performed. The coordinate is setcorresponding to each picture element of the image. The X coordinate isset along the horizontal direction of the image. The Y coordinate is setalong the vertical direction of the image. The X coordinate increases inthe right direction, and the Y coordinate increases downward.

FIG. 30 is a flowchart showing an embodiment of reducing an input image.

In FIG. 30, an original image is input in step S121.

Then, a range of n horizontal picture elements×n vertical pictureelements from the left top point of the original image (left topcoordinate (1, 1), right bottom coordinate (X, Y)) is set in step S122.

It is determined in step S123 whether or not black picture elementsexist in the determined range of the original image. If there are blackpicture elements in the determined range of the original image, thencontrol is passed to step S124 and the picture elements at thecoordinate (X/n, Y/n) of a reduced image are defined as black pictureelements. If there are no black picture elements in the determined rangeof the original image, then control is passed to step S125 and thepicture elements at the coordinate (X/n, Y/n) of a reduced image aredefined as white picture elements.

Then, it is determined in step S126 whether or not the process has beenperformed up to the right bottom of the original image. If the processhas not been performed up to the right bottom of the original image,then control is passed to step S127, and it is determined whether or notthe process has been performed up to the rightmost portion.

If the process has not been performed up to the rightmost portion, thena range of n horizontal picture elements×n vertical picture elements(left top coordinate (x, y), right bottom coordinate (X, Y)) is set tothe right of the processed range. If the process has been performed upto the rightmost portion, then a range of n horizontal picture elementsx n vertical picture elements (left top coordinate (x, y), right bottomcoordinate (X, Y)) is set below the processed range and to the right ofthe original image. Then, control is returned to step S123, and theabove described processes are repeated until the reducing process hasbeen completed on the entire range of the original image.

It is determined whether or not a character touches a character box byperforming a searching process on compressed image data processed in aninput image reducing process along and inside the lines forming part ofa character box. A rectangular area is enlarged at the side touching thecharacter outside by a predetermined distance. The coordinate of theenlarged rectangular area is converted into the coordinate of theoriginal image data.

For example, assume that a range 110 of a character box for thecompressed image data is extracted, a character 112 of ‘4’ exists in therectangular area encompassed by the character box, and the character 112of ‘4’ touches a lower side 111 of the character box as shown in FIG.31A.

As shown in FIG. 31B, a searching process is performed straight alongthe inside of the character box. If the search line crosses any pattern,it is assumed that a character exists near the character box, thecharacter possibly touches the character box, and that the character 112of ‘4’ in the rectangular area encompassed by the character box touchesthe character box. In this example, the character 112 of ‘4’ is assumedto touch the lower side 111 of the character box.

Then, the searching process is performed along the inside of the side111 of the character box. When the character 112 is regarded as touchingthe side 111 of the character box, the rectangular area enclosed by thesides of the character box is enlarged outward from the side 111 of thecharacter box touching the character 112 as shown in FIG. 31C. Anenlarged rectangular area 113 is defined as a character area containingthe character 112. If it is assumed that the character does not touchthe side of a character box, the portion enclosed by the character boxis assumed to be a character area.

Then, the coordinates of the rectangular area 113 shown in FIG. 31C isconverted into the coordinate in the original image data to obtain thecharacter area in the original image data from the character area in thecompressed image data. Thus, a rectangular area 116 can be obtained inthe original image data as shown in FIG. 31D.

A projecting process is performed on a side 114 of the character box inthe rectangular area 116, and the coordinate of the side 114 is computedfrom the original image data. At this time, the side 114 of thecharacter box is represented by a rectangle in the form of apredetermined length of strip. As shown in FIG. 31E, the pattern in therectangular area 116 is transmitted to be processed in a charactercompleting process, and a completing process is performed on a character115 touching the side 114 of the character box based on the coordinateof the side 114 of the character box computed according to the originalimage data.

FIG. 32 is a flow chart showing an embodiment of the process ofdetermining the existence of a character touching a character box.

In FIG. 32, a rectangle is represented in compressed image data in stepS131 in, for example, the process shown in FIG. 30.

A rectangular portion encompassed by four vertical and horizontal linesis extracted in step S132.

Then, the coordinates indicating the left top corner and right bottomcorner inside the rectangle are computed in step S133.

In step S134, the compressed image is searched for along the four sides(upper horizontal side, lower horizontal side, right vertical side, andleft vertical side) of the rectangle inside the character box.

If one of the four sides crosses the image pattern during the searchingprocess in step S135, then it is assumed that a character touches theside currently being searched.

The coordinates of the rectangle inside the character box are convertedinto the coordinates in the original image data in step S136 so that arectangular area in the original image data can be computed from therectangular area in the compressed image data.

In step S137, the rectangular area computed in step S136 is defined as acharacter area in the original image data.

It is determined in step S138 whether or not a character touches thecharacter box in the process in step S135. When a character touches thecharacter box, the box touching character range obtaining process isperformed in steps S139 through S143.

In the box touching character range obtaining process, the characterarea is enlarged outward from the side touching a character in stepS139, and the position at a predetermined distance outside the positionof the character area computed in step S137 is defined as the end of thecharacter area.

In step S140, the coordinate of the position of the side of thecharacter box in the original image data is computed from the coordinateof the position of the side of the character box in the compressed imagedata by converting the coordinate of the position of the side of thecharacter box contained in the character area computed in step S139 intothe coordinate in the original image.

In step S141, a projecting process is performed in the horizontaldirection for horizontal sides and in the vertical direction forvertical sides on the character box area in the original image dataobtained based on the coordinate of the position of the character box inthe original image data computed in step S140.

Next, in step S142, it is assumed that the area indicating projectionvalues larger than a predetermined value refers to the coordinates ofthe character box in the original image.

Then, in step S143, the computed coordinate of the character area in theoriginal image and the coordinate indicating the position of thecharacter box in the character area are transmitted to the charactercompleting process.

In step S144, the computed coordinate of the character area in theoriginal image is defined as a character area.

Described below are the correction feature extracting process in stepS41 and the correction character candidate extracting process in stepS42 shown in FIG. 12.

FIG. 33 shows an embodiment of a corrected character.

In FIG. 33, a character is corrected with deletion lines. A charactercan be deleted with ‘x’ as shown in FIG. 33A; with double horizontallines as shown in FIG. 33B; with diagonal lines as shown in FIG. 33C;with random lines as shown in FIG. 33D; and by painting black as shownin FIG. 33E.

The features of a deleted character can be extracted from the abovedescribed deleted character. The features can be the line density in apredetermined direction, an Euler number, and the density of blackpicture elements.

The ‘line density in a predetermined direction’ is obtained by countingthe changes from white picture elements into black picture elements (orblack picture elements into white picture elements) while an image in arectangular area is scanned in a predetermined direction. Thepredetermined direction refers to the direction vertical to the linepredicted as a deletion line.

For example, FIG. 34A shows an example of counting the maximum linedensity about the character ‘6’ in the vertical direction. In thisexample, the maximum line density in the vertical direction is 3.

The ‘line density in a predetermined direction’ of a deleted charactertends to increase as compared with the ‘line density in a predetermineddirection’ of a normal character. Computing the ‘line density in apredetermined direction’ extracts a candidate for a deleted character.

An Euler number ‘E’ is obtained by subtracting the number H of holes inan image from the number C of the elements linked in the image.

For example, FIG. 34B shows an example of two elements linked in animage and only one hole in the image. The Euler number E in this exampleis E=C−H=2−1=1.

The Euler number of a corrected character tends to be a negative numberindicating a large absolute number while the Euler number of a normalcharacter tends to be a number (2˜−1) indicating a small absolutenumber. Therefore, computing the Euler number extracts a candidate for adeleted character.

The density D of black picture elements refers to a ratio of the area B(number of black picture elements) of an object image to the area S ofthe rectangle circumscribing the object image.

For example, FIG. 34C shows an example of the case where the density Dof black picture elements is computed about the character ‘4’. Assumingthat S indicates the area of a rectangle circumscribing the character‘4’, and B indicates the area of the character ‘4’, the equation D=B/Sis expressed.

The ‘density of black picture elements’ of a deleted character tends tobe higher than the ‘density of black picture elements’ of a deletedcharacter. Computing the ‘density of black picture elements’ extracts acandidate for a deleted character.

Described below in the concrete is the basic character recognizing unit17 shown in FIG. 7.

FIG. 35 is a block diagram showing an embodiment of the configuration ofthe basic character recognizing unit 17.

In FIG. 35, a feature extracting unit 121 extracts the features of acharacter from an unknown input character pattern, and represents theextracted features by the feature vectors. On the other hand, a basicdictionary 122 stores the feature vectors of each character category.

A collating unit 123 collates the feature vector of an unknown characterpattern extracted by the feature extracting unit 121 with the featurevector of each character category stored in the basic dictionary 122,and computes the distance D_(ij) (i indicates the feature vector of theunknown character, and j indicates the feature vector of the category ofthe basic dictionary 122) between feature vectors in a feature space. Asa result, the category j indicating the shortest distance D_(ij) betweenthe feature vectors as an unknown character i.

The distance D_(ij) between the feature vectors in the feature space canbe computed using a Euclidean distance Σ (i−j)², a city block distanceΣ|i−j|, or an identification function such as a discriminant function.

Assuming that the distance from the first category is D_(ij1) and thedistance from the second category is D_(ij2), a table 1 arepreliminarily generated using the first category j1, the second categoryj2, the distance (D_(ij2)−D_(ij1)), and the reliability between thecategories. Additionally, a second table 2 is also generated using thedistance from the first category D_(ij1), the first category j1, and thereliability. The smaller reliability obtained from table 1 or 2 isstored in the intermediate process result table.

FIG. 36 shows an example of computing a feature vector.

In this example, the character ‘1’ is written in a column of 20 blocks(5 in vertical direction and 4 in horizontal direction). A black-paintedblock indicates ‘1’ and a white-painted block indicates ‘0’. The blocksare checked sequentially from the left-top block to the right-bottomblock. The characters ‘1’ or ‘0’ are rearranged as feature vectors.

For example, the feature vectorA indicated by (B) in FIG. 36 isrepresented by vector A=(1,1,1,1,0,0,0,1,1,1,1,1,1,0,0,0,1,1,1,1,). Thefeature vectorB indicated by (C) in FIG. 36 is represented by vectorB=(0,1,1,1,0,0,0,1,1,1,1,1,1,0,0,0,1,1,1,1,). The feature vectorCindicated by (D) in FIG. 36 is represented by vectorC=(1,1,1,1,0,0,0,1,0,1,1,0,1,0,0,0,1,1,1,1,).

FIG. 37 shows an example of computing the distance D_(ij) between thefeature vectors using the city block distance d (i, j).

Assuming that N indicates the number of dimensions of a feature vector,and i indicates the number of the feature vector, the i-th featurevector x_(i) is represented as x_(i)=(x_(i1), X_(i2), X_(i3), . . . ,x_(iN)), and the j-th feature vector x_(j) is represented as x_(j) =(x_(j1), x_(j2), x_(j3), . . . , x_(jN)) in the city block d (i, j). Thecity block distance d (i, j) between the i-th feature vector x_(i) andthe j-th feature vector x_(j) is defined as follows.

d(i, j)=|X _(i) −X _(j)|  (2)

For example, in FIG. 37, the basic dictionary 122 contains the featurevectors of the character categories of ‘1’, ‘2’, ‘3’, and ‘4’. Thefeature vector1 of the character category of ‘1’ is represented asvector1 =(0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,). The feature vector2of the character category of ‘2’ is represented as vector2 =(1,1,1,1,0,0,0,1,1,1,1,1,1,0,0,0,1,1,1,1,). The feature vector3 of thecharacter category of ‘3’ is represented asvector3=(1,1,1,1,0,0,0,1,1,1,1,1,0,0,0,1,1,1,1,1,). The feature vector4of the character category of ‘4’ is represented asvector4=(1,0,1,0,1,0,1,0,1,1,1,1,0,0,1,0,0,0,1,0,).

If an unknown character having the feature vector isvector=(0,1,1,1,0,0,0,1,1,1,1,1,1,0,0,0,1,1,1,1,) is input, then thecity block distances d (i, j) between the feature vector and each of thefeature vector1 of the character category of ‘1’, the feature vector2 ofthe character category of ‘2’, the feature vector3 of the charactercategory of ‘3’, and the feature vector4 of the character category of‘4’ entered in the basic dictionary 122 is computed by equation (2)above.

That is, the city block distance d (i, j) between the vector of theunknown character and the feature vector1 of the character category of‘1’ is represented as d(i, j) = | vector − vector1 | = | 0 − 0 | + | 1 −1 | + | 1 − 1 | + | 1 − 0 | + | 0 − 0 | + | 0 − 1 | + | 0 − 1 | + | 1 −0 | + | 1 − 0 | + | 1 − 1 | + | 1 − 1 | + | 1 − 0 | + | 1 − 0 | + | 0 −1 | + | 0 − 1 | + | 0 − 0 | + | 1 − 0 | + | 1 − 1 | + | 1 − 1 | + | 1 −0 | = 11

Similarly, the city block distance d (i, j) between the vector of theunknown character and the feature vector2 of the character category of‘2’ is represented as d(i, j) = | vector − vector2 | = 1. The city blockdistance d (i, j) between the vector of the unknown character and thefeature vector3 of the character category of ‘3’ is represented as d(i,j) = | vector − vector3 | = 3. The city block distance d (i, j) betweenthe vector of the unknown character and the feature vector4 of thecharacter category of ‘4’ is represented as d(i, j) = | vector − vector4| = 11.

Of the city block distances d (i, j) between the feature vector and eachof the feature vector1 of the character category of ‘1’, the featurevector2 of the character category of ‘2’, the feature vector3 of thecharacter category of ‘3’, and the feature vector4 of the charactercategory of ‘4’, the city block distances d (i, j) between the featurevector and the feature vector2 of the character category of ‘2’indicates the minimum value.

Therefore, it is determined that an unknown character whose featurevector =(0,1,1,1,0,0,0,1,1,1,1,1,1,0,0,0,1,1,1,1,) belongs to thecharacter category of ‘2’.

Described next is the method of identifying the details stored in theknowledge table 18 of the basic character recognizing unit 17 shown inFIG. 7. In this method of identifying the details, a local partialpattern is extracted as a character segment, and the change in positionand angle of the character segment of an unknown character is comparedwith the change in position and angle of the character segmentpreliminarily stored in the segment dictionary. Thus, the character canbe recognized by making the unknown character correspond to charactercategories.

FIG. 38 shows the method of extracting a character segment.

FIG. 38A shows a binary image pattern about the character ‘2’, and theportion with diagonal lines refers to a character represented by blackpicture elements.

FIG. 38B shows the outline of the character extracted from the binaryimage pattern shown in FIG. 38A. The dotted-line portion indicates theoriginal binary image pattern.

FIG. 38C shows the outline shown in FIG. 38B into the character segmentsS1 and S2 and the end portions T1 and T2. The end portions T1 and T2correspond to the start and end of the character ‘2’ shown in FIG. 38A.

FIG. 39 shows the method of detecting an end point.

In FIG. 39, the end point is detected as a portion where the slope ofthe outline indicates a sudden change. Actually, the three points A, B,and C at predetermined intervals are set on the outline S. The areamaking an angle of θ smaller than a predetermined value at the point Aon the outline between the points B and C is detected as an end portion.

When a character segment is extracted from a binary image pattern bydividing an outline of a character at the end portion, therepresentative points X, Y, and Z are set, for example, at predeterminedintervals on the character segment. The angles made at the consecutiverepresentative points X, Y, and Z are obtained, and an accumulativevalue of angle changes from the first representative point to each ofthe other representative points on the character segment is obtained asthe features at the representative points X, Y, and Z.

FIG. 40 shows the method of detecting a change in angle.

In FIG. 40, the representative points X, Y, and Z are set atpredetermined intervals on the outline S. The vector XY from therepresentative point X to the representative point Y and the vector YZfrom the representative point Y to the representative point Z aregenerated. The angle θ₂ between the vectors XY and YZ is defined as achange in angle at the representative point Y.

The change in angle at the representative point X on the outline S as aninitial value of a change in angle refers to the angle θ₁ made by thevector GX from the center of the gravity G of a character to therepresentative point X and the vector XY.

The feature at each of the representative points X, Y, and Z isrepresented by an accumulative value of changes in angle from therepresentative point X having the initial value of a change in angle toeach of the representative points Y and Z. For example, the feature atthe representative point Y is expressed as θ₁ + θ₂.

After obtaining the accumulative value of changes in angle at therepresentative point on a character segment of an unknown character, therepresentative point of a character segment of the unknown character ismade to correspond to the representative point on the character segmentstored in the segment dictionary. That is, the distance between theaccumulative value of changes in angle at the representative point onthe character segment of the unknown character and the accumulativevalue of changes in angle of the representative point on the charactersegment stored in the segment dictionary is computed. The representativepoint on the character segment in the segment dictionary indicating theshortest distance is made to correspond to the representative point of acharacter segment of the unknown character.

FIG. 41A shows the correspondence between the representative point ofthe character segment on an unknown character and the representativepoint on the character segment of the segment dictionary.

In FIG. 41A, the representative points a₁ through a₈ refer to therepresentative points on the character segment of an unknown character.The representative points b₁ through b₈ refer to the representativepoints on the character segment stored in the segment dictionary. Eachof the representative points a₁ through a₈ on the character segment ofthe unknown character corresponds to one of the representative points b₁through b₈ on the character segment stored in the segment dictionary.

After obtaining the correspondence between the representative points onthe character segment of an unknown character and the representativepoints on the character segments of the segment dictionary, therepresentative point on the character segment of the unknown charactercorresponding to the reference point on the character segment stored inthe segment dictionary is set as a check point.

FIG. 41B shows the correspondence between the reference point and thecheck point.

In FIG. 41B, the reference points d₁ and d₂ of the character segmentstored in the segment dictionary correspond respectively to the checkpoints c₁ and c₂ of the character segment of the unknown character.

After obtaining the correspondence between the reference points and thecheck points, the check information about the check points c₁ and c₂ ofthe character segment of the unknown character is computed.

The check information can be, for example, absolute positionalinformation about an individual check point as to where the check pointexists in the entire character image; relative positional informationabout two check points indicating the distance, direction, etc. betweenthe two check points; and information on two or more check points aboutthe changes in angle, linearity, etc. among the check points.

If predetermined conditions are satisfied after computing the checkinformation on the check points, then the character category of thecharacter segment which satisfies the condition and is stored in thesegment dictionary is output as a recognition result of the unknowncharacter.

For example, using the change in angle from the check point c₁ to thecheck point c₂ on the character segment shown in FIG. 41B along thecharacter segment as the check information under the determinationconditions, the character pattern shown in FIG. 38A can be recognized asbelonging to the character category of ‘2’ by computing the change inangle from the check point c₁ to the check point c₂ on the charactersegment shown in FIG. 41B along the character segment when a characterimage of the character segment indicating the change in angle of 60degrees or more belongs to the character category of ‘2’ in the segmentdictionary stored corresponding to the character segment.

FIG. 42 is a flowchart showing the character recognizing process by thedetail identifying method.

In FIG. 42, a list, for example, to be character-recognized is scannedby a scanner, and the read character image is binarized into a binarymonochrome image in step S150.

Then, in step S151, a character segment is extracted from the binaryimage obtained in step S150.

In step S152, a character segment which is not associated with thecharacter segment of an unknown character is detected from a pluralityof character segments stored in the segment dictionary.

In step S153, the character segment retrieved from the segmentdictionary is associated with the character segment of the unknowncharacter.

In step S154, a check point is selected from the representative pointson the character segment of the unknown character, and the checkinformation about the check point is computed.

In step S155, the character segment retrieved from the segmentdictionary is compared with the character segment of the unknowncharacter according to the check information computed in step S154, anda candidate for the character corresponding to the unknown character isdetermined by determining whether or not the check information about thecharacter segment retrieved from the segment dictionary matches thecheck information about the character segment of the unknown character.

Then in step S156, when a candidate for the character is determined inthe process of determining a candidate for the unknown character, thecharacter category corresponding to the character segment retrieved instep S153 is output as a recognition result. If a candidate for thecharacter is not determined, control is passed to step S157, and it isdetermined whether or not there is an unprocessed character segmentwhich is not associated with the character segment of the unknowncharacter. If there is an unprocessed character segment in the segmentdictionary, then control is returned to step S152 and the abovedescribed process is repeated.

If there is not an unprocessed character segment, which is notassociated with the character segment of an unknown character, in thesegment dictionary, then it is determined that the input unknowncharacter cannot be recognized, and ‘unrecognized’ is output as therecognition result.

The detail identifying method is disclosed by Tokukaihei 6-309501.

Described below is an embodiment of the box-touching characterrecognizing unit 13 shown in FIG. 9.

FIG. 43 shows the character completing process performed by thebox-touching character recognizing unit 13.

In this character completing process, only a character box is extractedfrom the binary image of the box-touching character. At this time, thebox-touching portion of the character stroke touching the character boxfor the box-touching character becomes obscure, and the character strokeis divided into a plurality of portions. Therefore, the geometricstructure of labelled character strokes such as the distance, direction,etc. of the divided portions of the character is evaluated, and thecharacter is completed.

For example, as shown in FIG. 43A, label 1 is assigned to the binaryimage of the character pattern 131 representing ‘3’ and the characterbox 132 touching the character. Then, the character pattern 131indicating ‘3’ is divided into 3 portions as shown in FIG. 43B byextracting the character box 132 from the binary image shown in FIG.43A, and by removing the character box 132. Thus, the three characterstrokes are generated with labels 1, 2, and 3 assigned.

The geometric structure such as the distance, direction, etc. of thethree labelled character strokes assigned labels 1, 2, and 3 isevaluated and the character is completed. Thus, the three characterstrokes assigned labels 1, 2, and 3 are linked, and a charactercompleted pattern 132 indicating ‘3’ with label 1 is generated as shownin FIG. 43C.

A recognizing process is performed on the character restored in thecharacter completing process as a candidate for a character to berecognized. In this recognizing process, the character is collated withthe standard pattern entered in the character category dictionary, and acode of a character category indicating the smallest difference isoutput.

FIG. 44 shows the re-completing process performed by the box-touchingcharacter recognizing unit 13.

In this re-completing process, if the character stroke parallel with thecharacter box touches the character box and is removed when thecharacter box is removed, then the character stroke is interpolated. Thebox-touching character is extracted based on the linkage using thelabels, and the character stroke parallel with the character box can beinterpolated by detecting the matching in linkage between the completedcharacter pattern completed by the character completing process and thebox-touching character.

For example, a binary image of a character pattern 141 indicating ‘7’touching a character box 142 as shown in FIG. 44A is assigned label 1.The character box 142 is extracted from the binary image shown in FIG.44A, and the character box 142 is removed. As shown in FIG. 44B, thecharacter pattern 141 indicating ‘7’ is divided into 3 portions andthree character strokes are generated with labels 1, 2 and 3 assigned.

The geometric structure such as the distance, direction, etc. of thethree labelled character strokes assigned labels 1, 2, and 3 isevaluated and the character is completed Thus, the three characterstrokes assigned labels 1 and 2 are linked, and a character completedpattern 142 formed of two character strokes assigned with labels 1 and 2is generated as shown in FIG. 44C.

In the character completing process, the character is completed onlybetween the portion assigned label 1 and the portion assigned label 2 asshown in FIG. 44B. However, the character is not completed between theportion assigned label 1 and the portion assigned label 3 as shown inFIG. 44B. The character is completed in the re-completing processbetween the portion. assigned label 1 and the portion assigned label 3as shown in FIG. 44B.

In the re-completing process, a character stroke parallel with thecharacter box is interpolated by preliminarily extracting thebox-touching character based on the linkage using labels, and bydetecting the matching in linkage between the pattern shown in FIG. 44Cand the box-touching character. That is, the patterns assigned labels 1and 2 as shown in FIG. 44C have been linked before removing thecharacter box as shown in FIG. 44A. Therefore, the patterns assignedlables 1 and 2 as shown in FIG. 44C are linked using the characterstroke parallel with the character box.

Thus, the binary image ‘7’ divided into two character strokes assignedlables 1 and 2 as shown in FIG. 44C can be completed, and a re-completedpattern 143 indicating ‘7’ with label 1 is generated as shown in FIG.44D.

The recognizing process is performed on the character restored in thecharacter completing process as a candidate for a character to berecognized. In this recognizing process, the character is collated withthe standard pattern entered in the character category dictionary, and acode of a character category indicating the smallest difference isoutput.

That is, in the example shown in FIG. 44, the character completedpattern 142 shown in FIG. 44C is recognized as belonging to the categoryof the character ‘’. The character completed pattern 143 shown in FIG.44D is recognized as belonging to the category of the character ‘7’.After it is determined that ‘7’ indicates a smaller difference than ‘’the character is finally recognized as ‘7’ and the character code isoutput.

Described below is the case where the recognizing process is performedby the box-touching character recognizing unit 13 shown in FIG. 7 byreferring to the knowledge table 14.

FIG. 45 shows an example of recognizing a box-touching character bylearning a pair of a character and its misread character and entering itin the knowledge table 14.

In this example, as shown in FIG. 45A, label 1 is assigned to the binaryimage of the character pattern 151 representing ‘2’ and the characterbox 152 touching the character. Then, the character pattern 151indicating ‘2’ is divided into 2 partial patterns with lables 1 and 2 asshown in FIG. 45B by extracting the character box 152 from the binaryimage shown in FIG. 45A, and by removing the character box 152.

As shown in FIG. 45C, the two partial patterns with lables 1 and 2 shownin FIG. 45B are linked and the character completed pattern 153 isgenerated in the character completing process.

In this case, the lower stroke of the character pattern 151 indicating‘2’ touches the lower side of the character box 152, and the touchingportion of the character almost completely overlaps the character box152. Therefore, even the re-completing process cannot complete the lowerstroke of the character pattern 151 indicating ‘2’, and there is a highpossibility that the character ‘2’ can be misread as ‘7’.

Thus, the box-touching character is not partly written outside thecharacter box, but completely overlaps the character box. Therefore, ifit can be misread as any other character, the box-touching charactershould be correctly recognized by entering it through learning a pair ofa character and its misread character.

Described below is the method of recognizing a box-touching character byentering a pair of the character and its misread character.

FIG. 46 is a block diagram showing the configuration for learning a pairof a character and its misread character in the box-touching characterrecognizing unit 13 shown in FIG. 7.

An automatic box-touching character generating unit 161 generates abox-touching character by making a character box overlap a learningcharacter input as not touching the character box. By the method ofchanging the learning character relative to its character box, aplurality of box-touching characters can be generated for a singlelearning character. In FIG. 46, a learning character 168 indicating ‘2’is input to the automatic box-touching character generating unit 161,and a box-touching character 169 is generated with the lower stroke ofthe character ‘2’ overlapping the lower side of the character box. Theinformation generated by the automatic box-touching character generatingunit 161 is entered in a knowledge table 167.

There can be two variations of characters with a learning characteroverlapping its character box, that is, a variation of a characterrelative to its character box, and a variation of a character box. Thevariation of a character relative to its character box can be, forexample, a displacement, a variation in size, a variation in pose, etc.The variation of a character box can be, for example, a variation inpose, a variation in width of a character box, a variation in size, aconvexity and concavity of a character box, etc.

The following parameters indicates the amount of a change in each of theabove described variations. The x axis indicates the vertical directionand the y axis indicates the horizontal axis.

1. Variation of a Character Relative to Its Character Box

Displacement: dx, dy

where dx (position indicated by black dot in FIG. 47) and dy (positionindicated by x in FIG. 47) respectively indicate the difference inposition of the center of gravity of the character and the character boxbetween the x and y directions.

Variation in size: dsx, dsy

where dsx and dsy indicate the size of a character in the x and ydirections respectively.

Variation in pose: dα

where dα indicates the angle of the pose of the character to thevertical line.

2. Variation in character box

Variation in pose: fα

where fα indicates the angle of the pose of the character box to thevertical line.

Variation in width of character box: w

where w indicates the width of the character box.

Variation in size: fsx, fsy

where fsx and fsy indicate the size of the character in the x and ydirections respectively.

Convexity and concavity of character box: fδ

where fδ is a parameter for controlling, for example, the concavity andconvexity of a character box with the deterioration, etc. of the qualityof a printed character box on a facsimile taken into account. Assumingthat the circumference of the character box is represented by L, fδ isrepresented by the array fδ[L] in size L. In this array, each element fδ[i] (i=1, 2, 3, . . . ) is an integer in the range of −β˜+β determinedby a random number.

Based on the above described type and amount of variation, abox-touching character is generated by providing a learning characterwith an operation F(dx, dy, dsx, dsy, dα, w, fsx, fsy, fα, fδ).

FIG. 47 shows an example of generating a box-touching character byassigning a character box 172 to a learning character 171 indicating‘7’.

As indicated by (A) shown in FIG. 47, providing a changing operation F(dx, dy, dsx, dsy, dα, w, fsx, fsy, fα, fδ) for the learning character171 indicating ‘7’ generates a box-touching character ‘7’ touching thecharacter box 172 as indicated by (B) shown in FIG. 47.

That is, a box-touching character can be generated by performing thechanging operation F (dx, dy, dsx, dsy, dα, w, fsx, fsy, fα, fδ) for thelearning character 171 and the character box 172 to make the learningcharacter 171 and the character box 172 overlap each other. At thistime, the changing operation F (dx, dy, dsx, dsy, dα, w, fsx, fsy, fα,fδ) is performed while, for example, fixing the position of the centerof gravity of the character box 172.

FIG. 48 shows an example of each type of a box-touching charactergenerated for the learning character ‘3’ with the size variation fsx inthe x direction and the size variation fsy in the y direction fixed andthe size of character box fixed.

(A) in FIG. 48 indicates an example of the type of variation‘displacement’, where the amount of the change is dx=0, and dy>0. Inthis case, the character ‘3’ is partly outside the lower side of thecharacter box (lower displacement).

(B) in FIG. 48 indicates an example of the type of variation‘displacement’, where the amount of the change is dsx=fsx, and dsy=fsy.In this case, the character ‘3’ touches the upper, lower, left, andright sides of the character box ‘3’. The rectangle circumscribing ‘3’equals the character box.

(C) in FIG. 48 indicates an example of the type of ‘variation in pose ofa character’, where the amount of the change is dα=10 degrees.

(D) in FIG. 48 indicates an example of the type of ‘variation in pose ofa character box’, where the amount of the change is fa=−10 degrees.

(E) in FIG. 48 indicates an example of the type of ‘variation in widthof a character box’, where the amount of the change is w=5.

(F) in FIG. 48 indicates an example of the type of ‘variation inconcavity and convexity of a character box’, where each element fδ[i] ofthe amount of change fδ[L] is controlled.

Next, a character box removing unit 162 shown in FIG. 46 extracts onlythe character box from the box-touching character generated by theautomatic box-touching character generating unit 161, and outputs to acharacter completing unit 163 the image data on the obscure characterobtained by removing the character box.

The character completing unit 163 evaluates and completes the geometricstructure such as the distance, direction, etc. of the labelledcharacter strokes on the image data from which the character box hasbeen removed by the character box removing unit 162. FIG. 46 shows anexample of generating a character completed pattern 170 by completing abox-touching character by the character completing unit 163 afterremoving the character box from the box-touching character 169 generatedby the automatic box-touching character generating unit 161.

A re-completing unit 164 preliminarily extracts the box-touchingcharacter based on the linkage using labels in the area where thecharacter completing unit 163 has. not completed the image data, andcompletes a character stroke parallel with the character box bydetecting the matching in linkage between the pattern completed by thecharacter completing unit 163 and the box-touching character.

The character completed pattern completed by the character completingunit 163 and the re-completed pattern completed by the re-completingunit 164 are input to a basic character recognizing unit 165.

The basic character recognizing unit 165 performs a characterrecognizing process on the character completed pattern completed by thecharacter completing unit 163 and the re-completed pattern re-completedby the re-completing unit 164. Then, the basic character recognizingunit 165 outputs the recognition result about each learning character toa character box touching state and recognition knowledge acquiring unit166.

The character box touching state and recognition knowledge acquiringunit 166 compares a recognition result output from the basic characterrecognizing unit 165 with preliminarily provided solution data andobtains the recognition ratio for the entire sample data. Then, thecharacter box touching state and recognition knowledge acquiring unit166 enters in the knowledge table 167 this recognition ratio asreliability, and the combination of the misread (mis-recognized)character and the correct character as a pair of a character and itsmisread character. The above described pair of a character and itsmisread character are entered in, for example, character codes. Thecharacter box touching state and recognition knowledge acquiring unit166 also extracts a parameter indicating the feature of the state of acharacter and a character box touching the character, and enters thefeature in the knowledge table 167.

Thus, the knowledge table 167 contains for each character category therecognition ratio, together with the pair of a character and its misreadcharacter, for the character in various touching states between thecharacter and the character box.

FIG. 49 shows an example of the knowledge table 167 generated throughlearning a character.

In FIG. 49, the knowledge table 167 contains, for example, a characterand its misread character (2 and 7) and the reliability of 77% togetherwith the amount of change dy=5 and W=5 in ‘displacement at a lowerposition’, etc. If the box-touching character ‘2’ indicates the amountof change dy=5 and W=5 in ‘displacement at a lower position’, then thebasic character recognizing unit 165 can mis-recognize ‘2’ for ‘7’ atthe probability of 23%. Even if the basic character recognizing unit 165misreads ‘2’ for ‘7’ in this case, it is determined by referring to theknowledge table 167 that the reliability is 77% and there is still theprobability of 22% that the character is actually ‘2’.

Similarly, the character box touching state and recognition knowledgeacquiring unit 166 enters in the knowledge table 167 the ‘amount ofchange’, ‘width of the sides of a character box’, a ‘character and itsmisread character’, and a reliability.

The pair (L1, L2) of a character and its misread character indicatesthat the character ‘L1’ may be actually mis-recognized as ‘L2’. Thecharacter codes for characters ‘L1’ and ‘L2’ are entered for thecorresponding characters ‘L1’ and ‘L2’.

In addition to the amount of change dy=5 and W=5 in displacement at alower position shown in FIG. 49, the knowledge table 167 also enterseach of the variations shown in FIG. 47 such as ‘variation in pose of acharacter relative to its character box’ (in this case, touching at theleft side) for each character category as shown in FIG. 50.

As shown in FIG. 50, dx=‘−3’˜‘+3’, dy=5, w=5, dsy=1, dα=‘−10’˜‘+10’,fα=‘−10’˜‘+10’ are entered for variations of ‘displacement at a lowerposition’. Thus, the amount of change entered in the knowledge table 167as ‘displacement at a lower position’ can be the displacement dx in thex direction, the displacement dy in the y direction, and other values.Similarly, dx=‘−3’˜‘+3’, dy=‘−3’˜‘+3’, w=5, dsy=1, dα=‘−20’˜‘+20’,fα=‘−10’˜‘+10’ are entered for a ‘variation of pose of a charactertouching the left side of the character box’.

A character recognizing method is followed on a pair (L1, L2) of acharacter and its misread character whose reliability is equal to orlower than a predetermined threshold (for example, 90%) in a way thatthe reliability is equal to or higher than the predetermined threshold.The learned character recognizing method is entered in the knowledgetable 167.

For example, the reliability of the character recognition of thebox-touching character ‘2’ in the state of ‘displacement at a lowerposition’ with dy=5 and w=5 is 77% as shown in FIG. 49. Since there is ahigh probability that the character is misread as ‘7’, the charactercompleted pattern completed by the character completing unit 163 or thecharacter re-completed pattern re-completed by the re-completing unit164 is entered in the knowledge table 167 after learning that, forexample, the recognition ratio can be improved by recognizing thepattern again by emphasizing a specified area.

The method of emphasizing a specified area for a pair (2 and 7) of acharacter and its misread character is described by referring to FIG.51.

First, as shown in FIG. 51A, a circumscribing rectangle 180circumscribing the character completed pattern completed by thecharacter completing unit 163 or the re-completed pattern re-completedby the re-completing unit 164 is divided into m×n divided areas in mcolumns by n rows. Then, with diagonal lines shown in FIG. 51B, theupper half m/2×n area of the circumscribing rectangle 180 is emphasizedto recognize the character again.

That is, the feature parameter of the m/2×n area is extracted, and it ischecked whether the character completed pattern completed by thecharacter completing unit 163 or the re-completed pattern re-completedby the re-completing unit 164 is ‘2’ or ‘7’. The area emphasizing methodimproves the recognition up to 95%. The knowledge table 167 shown inFIG. 49 contains, in the line of a pair of a character and its misreadcharacter (2, 7), the ‘highlighted area’ as a re-recognizing method, the‘m/2×n’ area as a re-recognition area, and ‘95%’ as the re-recognitionreliability.

The area emphasizing method is also effective for the box-touchingcharacter shown in FIG. 52A. FIG. 52A is an example that the lowerportion of the character pattern indicating ‘2’ touches a character box182.

In this case, the character completing unit 163 obtains a charactercompleted pattern 183 similar to ‘7’ shown in FIG. 52B. A circumscribingrectangle 184 shown in FIG. 52C is computed corresponding to thecharacter completed pattern 183. If, as shown in FIG. 51, thecircumscribing rectangle 184 is divided into m×n areas, and the upperhalf m/2×n partial area 185 is especially emphasized when thecharacter‘is recognized, then there is a high probability that thecharacter completed pattern 183 can be recognized as ‘2’. That is, it islearned that a solution (reliability) can be obtained at a high rate,and the above described area emphasizing method is entered in theknowledge table 167 as a re-recognizing method for the pair (2 and 7) ofa character and its misread character by touching a character box.

FIG. 53 is a flowchart showing the method of re-recognizing a characterpattern by emphasizing a specific area.

In FIG. 53, the data of a pair of a character and its misread characterindicating a lower reliability is retrieved from the knowledge table 167in step S161. Corresponding to the left character of the pair of thecharacter and its misread character, a character pattern as binarylearning data, and the character completed pattern completed by thecharacter completing unit 163 or the re-completed pattern re-completedby the re-completing unit 164 are input.

The character completed pattern or the re-completed pattern isprescribed by an amount-of-change parameter entered in the knowledgetable 167, and can be represented by plural forms of patterns even inthe same category.

The character pattern as the learning data input in step S161, and thecharacter completed pattern completed by the character completing unit163 or the re-completed pattern re-completed by the re-completing unit164 are divided into m×n areas in step S162.

IN step S163, a character recognizing process is performed on the X×Ypartial pattern in the m×n area, and the recognition ratio z isobtained.

The above described X×Y partial pattern is a re-recognition area where Xand Y indicates the length in the X and Y directions respectively in them×n area, and X≦m and Y≦n. The above described recognition ratio zindicates the probability that a correct solution can be output withcharacters recognized using the above described X×Y partial patterns.

That is, the character recognition result of the partial pattern of acharacter pattern as learning data is regarded as a solution. Thecharacter recognition result on plural partial patterns about thecharacter completed pattern completed by the character completing unit163 or the re-completed pattern re-completed by the re-completing unit164 is compared with the character recognition result on the partialpattern of the character pattern as learning data. As a result, therecognition ratio z of the partial pattern about the character completedpattern completed by the character completing unit 163 or there-completed pattern re-completed by the re-completing unit 164 isobtained.

Then, in step S164, it is discriminated whether or not the recognitionratio z is larger than the maximum recognition ratio max. The maximumrecognition ratio max is a variable storing the maximum value of therecognition ratio z obtained while a partial pattern of X×Y varies.First, an initial value is set (for example, ‘0’)

If the recognition ratio z is larger than the maximum recognition ratiomax, then control is passed to step S165 to substitute the recognitionratio z for the maximum recognition ratio max, and control is passed tostep S166 to check whether or not the lengths X and Y is variable. Ifthe recognition ratio z is equal to or smaller than the maximumrecognition ratio max in step S164, then control is immediately passedto step S166.

Changing the lengths X and Y is changing the size of the lengths X andY, and also includes a position change in the m×n area of the partialpattern of X×Y.

If it is discriminated in step S166 that the lengths X and Y arevariable, control is returned to step S163, the lengths X and Y arechanged, and a new partial pattern of X×Y is determined, and charactersare recognized in the partial pattern.

The processes in step S163 through S166 are repeated until it isdetermined in step S166 that the lengths X and Y cannot be changed. Ifit is determined in step S166 that the lengths X and Y cannot bechanged, the maximum identification ratio max and the partial pattern ofX×Y from which the maximum identification ratio max has been obtainedare entered in the knowledge table 167 as a re-recognition reliabilityand a re-recognition area respectively. The ‘area emphasis’ is enteredas a re-recognizing method in the knowledge table 167.

FIG. 53 is a flowchart showing an example of learning the method ofre-recognizing a character using the area emphasizing method. Thecharacter re-recognizing method can also be learned by any other methodsthan the area emphasizing method.

FIG. 54 is a block diagram showing the configuration with which abox-touching character is recognized according to the knowledge table167 obtained through learning.

In FIG. 54, a box-touching state detecting unit 191 detects the touchingstate between a character box and a character when an unknownbox-touching character is input. This example shows the lower portion ofa box-touching character pattern 201 (‘2’) partially overlapping thelower side of the character box as indicated by (A) shown in FIG. 54 andthe lower portion of a box-touching character pattern 203 (‘2’)completely overlapping the lower side of the character box as indicatedby (B) shown in FIG. 54. The box-touching state detecting unit 191detects the box-touching character pattern 201 and box-touchingcharacter pattern 203. A character box removing unit 192 removes acharacter box from the box-touching character pattern detected by thebox-touching state detecting unit 191.

A character completing unit 193 evaluates and completes the geometricstructure such as the distance, direction, etc. of the labelledcharacter strokes on the character pattern from which the character boxhas been removed by the character box removing unit 192.

A re-completing unit 194 preliminarily extracts the box-touchingcharacter based on the linkage using labels in the area where thecharacter completing unit 193 has not completed the image data, andcompletes a character stroke parallel with the character box bydetecting the matching in linkage between the pattern completed by thecharacter completing unit 193 and the box-touching character. Are-completed pattern 202 shows a pattern completed in the re-completingprocess performed by the re-completing unit 194 on the box-touchingcharacter pattern 201 indicated by (A) shown in FIG. 54. A re-completedpattern 204 shows a pattern which cannot be completed in there-completing process performed by the re-completing unit 194 on thebox-touching character pattern 203 indicated by (B) shown in FIG. 54.

The basic character recognizing unit 195 performs a characterrecognizing process on the character completed pattern completed by thecharacter completing unit 193 and the re-completed pattern re-completedby the re-completing unit 194. As a result, the character code of ‘2’ isoutput for the re-completed pattern 202 indicated by (A) shown in FIG.54, and the character code of ‘7’ is output for the re-completed pattern204 indicated by (B) shown in FIG. 54. The character codes obtained asthe recognition result are output to a character box touching state andrecognition knowledge acquiring unit 196.

The character box touching state and recognition knowledge acquiringunit 196 obtains the type of variation according to the positionalinformation about the rectangle circumscribing the character completedpattern completed by the character completing unit 193 or there-completed pattern re-completed by the re-completing unit 194, andaccording to the positional information and width information about thecharacter box extracted from the box-touching character pattern 201indicated by (A) shown in FIG. 54 or the box-touching character pattern203 indicated by (B) shown in FIG. 54.

That is, a change to a character relative to its character box such asthe displacement, variation in size, variation in pose, etc. as shown inFIG. 47, or a change to a character box such as a variation in pose,variation in width of a character box, convexity and concavity of acharacter box, etc. is obtained.

Furthermore computed is the amount of change dx, dy, dsx, dsy, dα, w,fsx, fsy, fα, or fδ for the type of each variation as obtained above.

Then, the knowledge table 167 is searched using, as key items, thecomputed variation type information, amount-of-change information, and acharacter code input from the basic character recognizing unit 195. Itis checked whether or not the knowledge table 167 stores a linecontaining the variation type information, amount-of-change.information, and a pair of a character and its misread charactermatching the key items.

If the line matching the key item exists, then it is discriminatedwhether or not the reliability stored in the line is equal to or largerthan a predetermined threshold. If it is smaller than the threshold,then the character completed pattern completed by the charactercompleting unit 193 or the re-completed pattern re-completed by there-completing unit 194 is output to a character re-recognizing unit 197.The characters are recognized again by the re-recognizing method enteredin the line.

That is, a box-touching character in unknown image data is re-recognizedby a method other than the method by the basic character recognizingunit 195 using the character completed pattern completed by thecharacter completing unit 193, the re-completed pattern re-completed bythe re-completing unit 194, or binary image data of an unknowncharacter. Then a character code obtained in the re-recognizing processis output.

For example, when the basic character recognizing unit 195 outputs thecharacter code of ‘7’ as a recognition result on a re-completed pattern204 re-completed by the re-completing unit 194, the character boxtouching state and recognition knowledge acquiring unit 196 obtains thetype of variation and an amount of change according to the positionalinformation about the rectangle circumscribing the re-completed pattern204, and the positional information and width information about thecharacter box extracted from the box-touching character pattern 203. Asa result, the ‘displacement at a lower position’ is computed as the typeof variation. ‘dy=5’ is obtained as the amount of change in the‘displacement at a lower position’. ‘w=5’ is computed as the width ofthe character box.

Then, the character box touching state and recognition knowledgeacquiring unit 196 searches the knowledge table 167 shown in FIG. 49using the ‘displacement at a lower position’ as the type of variation,‘dy=5’ as an amount of displacement at a lower position, ‘w=5’ as thewidth of a character, and the character code of ‘7’ received from thebasic character recognizing unit 195 as a key item. As a searchingresult, the lines corresponding to the key items store a pair (2 and 7)of a character and its misread character, and the reliability of thecharacter code ‘7’ recognized by the basic character recognizing unit195 is 77%, thereby referring to that the character ‘2’ is misread for‘7’ at the probability of 23%.

In this case, since the reliability entered in the lines correspondingto these key items is lower than a predetermined threshold, thecharacter re-recognizing unit 197 re-recognizes the box-touchingcharacter pattern 203 contained in the unknown image data by a methodother than the method followed by the basic character recognizing unit195. At this time, the character re-recognizing unit 197 refers to theline corresponding to the key item on the knowledge table 167 to specifythe re-recognizing method.

That is, the character re-recognizing unit 197 is informed of the ‘areaemphasizing method’ as a re-recognizing method, and of an upper halfm/2×n area 205 of the re-completed pattern 204 as a re-recognition areawhen the ‘area emphasizing process’ is performed. It is also informed ofthe re-recognition reliability of 95%.

The character re-recognizing unit 197 re-recognizes only the upper halfarea 205 of the re-completed pattern 204 by the re-recognizing methodentered in the knowledge table 167. The character re-recognizing unit197 is informed that the upper half area 205 of the re-completed pattern204 matches a partial area 207 of a character pattern 206 correspondingto the character code ‘2’ at a probability of 95%, and matches a partialarea 209 of a character pattern 208 corresponding to the character code‘7’ at a probability of 5%, and outputs the character code ‘2’ as arecognition result of the character touching the character box of thebox-touching character pattern 203 of an unknown character.

FIG. 55 is a flowchart showing the operations of the character boxtouching state and recognition knowledge acquiring unit 196.

In FIG. 55, an amount of change to a character relative to its characterbox is computed based on the character box extracted from an unknownbox-touching character pattern and the character pattern separated fromthe box-touching character pattern, and the knowledge table 167 issearched using the amount of change as a key item in step S171. Then, itis checked whether or not the knowledge table 167 stores a linecontaining the amount of change matching the computed amount of change.

When the amount of change to the character ‘2’ indicating thedisplacement at a lower position is obtained as, for example, dx=5 andw=5, the top line on the knowledge table 167 is detected as shown inFIG. 49.

If a line containing a matching amount of change exists, control ispassed to step S172, and it is determined whether or not the linecontaining the matching amount of change stores the line containing inthe pair of a character and its misread character the character code(character recognition code) input from the basic character recognizingunit 195.

The top line on the knowledge table 167 is detected with the character‘2’ indicating the displacement at a lower position as shown in FIG. 49.

In step S173, if the line containing the matching amount of changestores the line containing in the pair of a character and its misreadcharacter the character code input from the basic character recognizingunit 195, the re-recognition reliability entered in the correspondingline on the knowledge table 167 is compared with the reliabilitycomputed by the basic character recognizing unit 195. It is determinedwhether or not the re-recognition reliability entered in thecorresponding line on the knowledge table 167 is larger than thereliability computed by the basic character recognizing unit 195.

For the character ‘2’ displaced at its lower position, there-recognition reliability entered in the top line on the knowledgetable 167 shown in FIG. 49 and the reliability computed by the basiccharacter recognizing unit 195 are ‘95%’ and ‘77%’ respectively. Thus,it is determined that the re-recognition reliability entered in thecorresponding line on the knowledge table 167 is larger than thereliability computed by the basic character recognizing unit 195.

If the re-recognition reliability entered in the corresponding line onthe knowledge table 167 is larger than the reliability computed by thebasic character recognizing unit 195, control is passed to step S174 andit is determined whether or not the re-recognition reliability enteredin the corresponding line on the knowledge table 167 is larger than apredetermined threshold th1. If it is larger than the threshold th1,then control is passed to step S175, and the re-recognizing method andthe re-recognizing area entered in the line on the knowledge table 167and detected in step S172 are referred to.

Next, in step S176, a re-recognition area shown on the knowledge table167 is detected from the character completed pattern completed by thecharacter completing unit 193 or the re-completed pattern re-completedby the re-completing unit 194, and a character recognizing process isperformed on the detected area by the re-recognizing method shown on theknowledge table 167. Then, the character code obtained in the characterrecognizing process is output.

If the threshold th1 is lower than ‘95%’, then the character code of ‘2’is finally output by performing the character re-recognizing process bythe area emphasizing method using the upper half ‘m/2×n’ area on thecompleted pattern of the character ‘2’, which indicates the displacementat its lower position and is input by the basic character recognizingunit 195.

Described below is an embodiment of the character string recognizingunit 15 shown in FIG. 7.

The character string recognizing unit 15 does not determine in aheuristic manner the threshold when the character is integrated on theparameter as a feature value used when a character is detected one byone from a character string extracted in the layout analysis in step S2shown in FIG. 8. The threshold is determined as a statisticallyreasonable value.

Practically, a parameter value and a statistic data are obtained on thesuccessful or unsuccessful integration of a character corresponding tothe parameter. Each parameter is not individually evaluated, but iscounted as an element in a multiple-dimensional space, and thediscriminate phase is obtained by the multivariate analysis todiscriminate the two groups (cases) in the multiple-dimensional spacefor both cases where the integration is successfully performed andunsuccessfully performed.

That is, sample data comprising P feature values indicating the featuresof a pattern is divided into two groups, that is, a first group assuccessfully retrieved and a second group as unsuccessfully retrieved.The discriminant phase between the first and second groups is generatedin the P-dimensional space.

The discriminant phase can be obtained by, for example, the discriminantanalysis method. That is, when the discriminant phase is formed by alinear discriminant function, the coefficient vector of the discriminantfunction is expressed as follows. $\begin{matrix}{\Sigma^{- 1}\left( {\mu_{1} - \mu_{2}} \right)} & (3)\end{matrix}$

where

Σ indicates the variance co-variance matrix of the first and secondgroups;

μ₁ indicates the mean vector of the first group; and

μ₂ indicates the mean vector of the second group.

The discriminant function having the coefficient vector in equation (3)is generated in a way that an equal distance can be set from each centerof gravity of the first and second groups.

The coefficient vector of the discriminant function can also be computedbased on the standard that the ratio of the inter-group variationbetween the first and second groups to the intra-group variation can bethe largest possible.

The process of detecting a character from a character string isperformed separately by the statistic process of integrating characterpatterns by referring to the positions, sizes, arrays, etc. of thecircumscribing rectangles of character patterns, and by thenon-statistic process of observing the forms of character patterns tocorrectly process the superscript strokes, separate-stroke characters,etc.

In the statistic process, a detection parameter is used. The detectionparameter refers to the position and ratio of vertical to horizontalsize of a rectangle circumscribing a pattern, ratio of character size toan average character size, width of overlap between patterns, density ofcharacter string, etc.

Samples of detection parameters are listed below as shown in FIG. 56.

1) distance a between the right side of the circumscribing rectangle 211and the left side of the circumscribing rectangle 212

2) distance b between the left side of the circumscribing rectangle 211and the right side of the circumscribing rectangle 212

3) ratio c of the distance a between the right side of thecircumscribing rectangle 211 and the left side of the circumscribingrectangle 212 to the distance b between the left side of thecircumscribing rectangle 211 and the right side of the circumscribingrectangle 212

4) ratio d of the distance b between the left side of the circumscribingrectangle 211 and the right side of the circumscribing rectangle 212 toan average width MX of circumscribing rectangles

5) angle e made between the lower side of the circumscribing rectangle213 and the line connecting the mid-point of the lower side of thecircumscribing rectangle 213 to the mid-point of the lower side of thecircumscribing rectangle 214

6) angle f made between the lower side of the circumscribing rectangle213 and the line connecting the right-bottom vertex of thecircumscribing rectangle 213 to the left-bottom vertex of thecircumscribing rectangle 214

7) when the circumscribing rectangle 215 overlaps the circumscribingrectangle 216;

ratio g of the distance p between the right side of the circumscribingrectangle 215 and the left side of the circumscribing rectangle 216 tothe distance q between the left side of the circumscribing rectangle 215and the right side of the circumscribing rectangle 216.

That is,

 c=a/b  (4)

d=b/MX  (5)

g=p/q  (6)

The statistic process is described below by referring to the flowchartshown in FIG. 57.

In step S181, the rectangle circumscribing the link pattern isretrieved.

It is checked in step S182 whether or not there is anothercircumscribing rectangle to the right of the circumscribing rectangleretrieved in step S181. If there is no circumscribing rectangle to theright of the circumscribing rectangle retrieved in step S181, then thecircumscribing rectangle retrieved in step S181 is removed from theobjects of the statistic process.

If it is determined in step S182 that there is another circumscribingrectangle to the right of the circumscribing rectangle retrieved in stepS181, then control is passed to step S184.

In step S183, the average character size of the rectangle circumscribinga character string is computed. When the size of the rectanglecircumscribing a character string is computed, the exact averagecharacter size cannot be immediately computed because each character hasnot been detected yet.

For example, a provisional average character size is computed bytemporarily integrating the rectangle circumscribing a link pattern. Atemporary integration is performed when, for example, thevertical-horizontal ratio P of the integrated link pattern satisfies thefollowing equation.

N(=0.8)<P<M(=1.2)

An average character size is computed after the temporary integration.The average character size of a rectangle circumscribing a characterstring can also be obtained by generating a frequency histogram for eachsize of a circumscribing rectangle.

Then, the parameters a through g shown in FIG. 56 are computed in stepS184.

In the non-statistic process, separate-stroke characters and superscriptstrokes in a character string are processed respectively in theseparate-stroke-character process and the superscript-stroke process.

In the separate-stroke-character process, the pose and density of apattern, the size of an integrated pattern obtained by integratingadjacent patterns, and the distance between the patterns are used asdetection parameters.

For example, the following values are used as the detection parametersas shown in FIG. 58.

8) ratio p of the distance a between the right side of thecircumscribing rectangle 221 and the left side of the circumscribingrectangle 222 to the distance b between the left side of thecircumscribing rectangle 221 and the right side of the circumscribingrectangle 222

9) ratio q of the distance b between the left side of the circumscribingrectangle 221 and the right side of the circumscribing rectangle 222 toan average width MX of circumscribing rectangles

10) ratio r of the product of the area c of the circumscribing rectangle21 and the area d of the circumscribing rectangle 22 to the square ofthe product of the average width MX of circumscribing rectangles and theaverage height MY of the circumscribing rectangles.

That is,

p=a/b  (7)

q=b/MX  (8)

r=(c×d)/(MX×MY)²  (9)

The separate-stroke-character process is described below by referring tothe flowchart shown in FIG. 59. This separate-stroke-character processis to detect a separate-stroke character in a link pattern formed by twoor more separate strokes such as ‘’, ‘’, etc.

In step S191, it is determined whether or not a right-lifted patternexists in link patterns. If there is no right-lifted pattern, theseparate-stroke-character process is not performed.

If a right-lifted pattern is detected in step S191, then control ispassed to step S192 and it is determined whether or not there is aleft-lifted pattern adjacent to the right of the above describedright-lifted pattern, that is, a pattern of, for example, ‘’, or apattern adjacent to the right of the above described right-liftedpattern and intersecting another pattern (right angle line density) twotimes when it is searched for in the vertical direction, that is, apattern of, for example, ‘’. Unless the pattern refers to a pattern inthe form of ‘’ or ‘’ the separate-stroke character process is notperformed in this case.

If it is determined in step S192 the pattern refers to a pattern in theform of ‘’ or ‘’ control is passed to step S194.

In addition to steps S191 and S192, an average character size of acharacter string in a circumscribing rectangle is computed in step S193.

After the above described steps S192 and S193 have been completed, thevalues of the parameters p through r shown in FIG. 54 are computed instep S194.

In the superscript-stroke process, a candidate for a pattern withsuperscript strokes is checked to use the size of the adjacent patternsintegrated, the distance between these patterns, and the ratio of thesize of the characters to the size of an average character as detectionparameters.

That is, the following values are used as the detection parameters asshown in FIG. 56.

11) ratio p of the distance a between the right side of thecircumscribing rectangle 231 and the left side of the circumscribingrectangle 232 to the distance b between the left side of thecircumscribing rectangle 231 and the right side of the circumscribingrectangle 232

12) ratio q of the distance b between the left side of thecircumscribing rectangle 231 and the right side of the circumscribingrectangle 232 to an average width MX of circumscribing rectangles

13) ratio r of the product of the area c of the circumscribing rectangle231 and the area d of the circumscribing rectangle 232 to the square ofthe product of the average width MX of circumscribing rectangles and theaverage height MY of the circumscribing rectangles is used as adetection parameter.

That is, parameters p through r can be expressed as in equations (7)through (9).

The superscript-stroke process is described below by referring to theflowchart shown in FIG. 61.

First, in step S201, a pattern of a candidate for a superscript strokeis extracted. For example, when two adjacent link patterns are extractedby a link pattern extracting unit 1, and when the ratio of the size ofthe integrated pattern of the two adjacent patterns to an averagecharacter size of the circumscribing rectangle of a character string isequal to or smaller than a predetermined threshold, that is ¼, thepattern is extracted as a candidate for a pattern with superscriptstrokes.

It is checked in step S202 whether or not there is an adjacentcircumscribing rectangle to the left of the pattern which is a candidatefor a character with superscript strokes. Unless there is an adjacentcircumscribing rectangle to the left of the pattern which is a candidatefor a character with superscript strokes, then the candidate for acharacter with superscript strokes is removed from the objects to beprocessed in the superscript-stroke process.

If it is determined in step S202 that there is an adjacentcircumscribing rectangle to the left of the pattern which is a candidatefor a character with superscript strokes, then control is passed to stepS204.

In step S203, in addition to the above described steps S201 and S202, anaverage character size of a rectangle circumscribing a character stringis computed. When the processes in steps S202 and S203 are completed,the values of the parameters p through r shown in FIG. 60 are computedin step S204.

Next, a discriminant phase is set to compute the reliability indetecting a character from an unknown handwritten character string usinglearning data. If the number of parameters is n, then two groups aregenerated in the n-dimensional space to store in each group thecharacters detected successfully and unsuccessfully.

FIG. 62 is a flowchart showing the method of computing data onsuccessful and unsuccessful detection.

In FIG. 62, it is visually determined about the preliminarily collectedlearning data whether or not the object circumscribing rectangle and theadjacent circumscribing rectangle can be integrated into a singlecharacter in step S211. If the object circumscribing rectangle and theadjacent circumscribing rectangle can be integrated into a singlecharacter, then control is passed to step S212. If the objectcircumscribing rectangle and the adjacent circumscribing rectanglecannot be integrated into a single character, then control is passed tostep S213.

In step S212, when the object circumscribing rectangle and the adjacentcircumscribing rectangle can be integrated into a single character, thevalues of the parameters of the object circumscribing rectangle and theadjacent circumscribing rectangle are recorded. The parameters of theobject circumscribing rectangle and the adjacent circumscribingrectangle can be the parameters a through g shown in FIG. 52 in thestatistic process, and can be the parameters p through r shown in FIGS.58 and 60 in the non-statistic process.

In step S213, when the object circumscribing rectangle and the adjacentcircumscribing rectangle cannot be successfully integrated into a singlecharacter, the values of the parameters of the object circumscribingrectangle and the adjacent circumscribing rectangle are recorded.

Then, the values of the detection parameters in the statistic processand non-statistic process are computed about an unknown characterstring. A distance from a discriminant phase obtained from the learningdata is computed on the point in a multiple-dimensional space determinedby the value of the parameter. The obtained distance is quantified asthe detection reliability.

For example, when the number of feature parameters is 3, H indicates thediscriminant phase for use in discriminating the two groups, that is,successfully detected characters and unsuccessfully detected characters,and n indicates the unit normal vector of the discriminant phase H asshown in FIG. 63. If the value of a parameter equals the vector value ofp, the distance h between the discriminant phase and the point p in the3-dimensional space corresponding to the parameter value can beexpressed as follows.

h=OP·n  (10)

where OP is a vector from the origin O in the 3-dimensional space to thepoint p in the 3-dimensional space.

Whether the distance h from the discriminant phase H is positive ornegative determines to which group, that is, successfully detected groupor unsuccessfully detected group, the value of the parameter belongs,and determines to what extent the value of the parameter is away fromthe discriminant phase H.

As shown in FIG. 64, a successfully detected histogram 241 and anunsuccessfully detected histogram 242 are obtained for the entireparameters of the learning data in the multiple-dimensional space basedon the distance h from the discriminant phase H. Normally, since thehistogram distributions 241 and 242 are normal distributions, thehistogram distributions 241 and 242 are approximated at a normaldistribution. In these normal distributions, partially overlapping areasnormally exist.

According to the present invention, it is determined whether or not thepatterns are to be integrated in consideration of the reliability of thedetection of the adjacent pattern having a detection parameterpositioned at the overlapping area.

FIG. 65 is a flowchart showing an example of the method of computing thedetection reliability.

In FIG. 65, the distance h from the discriminant phase H to a point in amultiple-dimensional space determined by a plurality of parameters iscomputed by the above described equation (10) in step S221.

In step S222, the histogram distribution of a plurality of parametervalues obtained from the learning data is approximated using the normaldistribution. That is, as shown in FIG. 66, the histogram distributionof successfully detected patterns is approximated using a successfullydetected pattern normal distribution 251, and the histogram distributionof unsuccessfully detected patterns is approximated using anunsuccessfully detected pattern normal distribution 252.

In step S223, the overlap areas of the two groups is computed. Forexample, the overlap area between the successfully detected patternnormal distribution 251 and the unsuccessfully detected pattern normaldistribution 252 is computed as a 2-group overlap area 254 as shown inFIG. 62. At this time, an area 253 other than the 2-group overlap area254 in the successfully detected pattern normal distribution 251 is setas a successfully detected area. Similarly, an area 255 other than the2-group overlap area 254 in the unsuccessfully detected pattern normaldistribution 252 is set as an unsuccessfully detected area.

The position of the value of the parameter input for an unknowncharacter in the histogram distribution is determined in step S224.

In step S225, if the value of the parameter input for the unknowncharacter is included in the 2-group overlap area 254 as a determinationresult of the position of the value of the parameter input for theunknown character in the histogram distribution, then control is passedto step S226. Then, the detection reliability is computed based on thevalue of the parameter input for the unknown character in the 2-groupoverlap area 254.

If it is determined in step S225 that the value of the parameter inputfor the unknown character is not included in the 2-group overlap area254, then control is passed to step S226, and it is determined whetheror not the value of the parameter input for the unknown character isincluded in the successfully detected area 253.

If it is determined that the value of the parameter input for theunknown character is included in the successfully detected area 253,then control is passed to step S228, and the detection reliability isset to ‘1’. If it is determined that the value of the parameter inputfor the unknown character is not included in the successfully detectedarea 253, then control is passed to step S229, and the detectionreliability is set to ‘0’.

For example, if the distance from the discriminant phase to the value ofthe parameter input for the unknown character is included in the 2-groupoverlap area 254 as a result of computing the distance from thediscriminant phase to the value of the parameter input for the unknowncharacter, then the detection reliability is computed based on thedistance from the discriminant phase to the value of the parameter inputfor the unknown character. If the distance from the discriminant phaseto the value of the parameter input for the unknown character isincluded in the successfully detected area 253, then the detectionreliability is set to ‘1’. If the distance from the discriminant phaseto the value of the parameter input for the unknown character isincluded in the unsuccessfully detected area 255, then the detectionreliability is set to ‘0’.

FIG. 67 is a flowchart showing an example of computing the 2-groupoverlap area.

As shown in FIG. 67, an average value m and a distribution value v of ahistogram 261 is computed in step S231 about the histogram distributionof successfully detected patterns and unsuccessfully detected patternsobtained from the learning data.

The sum d of squares error between a normal distribution curve 262 andthe histogram 261 about the histogram 261 is computed in step S232 aboutthe histogram distribution of successfully detected patterns andunsuccessfully detected patterns.

In step S233, the adaptability T is computed by the following equation(11).

T=d/S  (11)

where S indicates the area of the normal distribution curve 262.

In step S234, the distance L from the center to the end of the normaldistribution curve 262 is computed by the following equation (12)

L=k×(1+T)×v ^(l/2)  (12)

where k indicates a constant of proportionality, and v^(1/2) equals astandard deviation.

In step S235, the area from a right end 267 of a normal distributioncurve 263 to a left end 266 of a normal distribution curve 264 is set asa 2-group overlap area 265.

Next, it is determined whether or not a recognizing process is performedbased on the detection reliability obtained in the process shown in FIG.65. In this case, for example, the recognizing process is not performedon a candidate for a detected character having high detectionreliability, but is performed on a candidate for a detected characterhaving low detection reliability.

For a plurality of candidates for detected characters, a character to bedetected is selected in consideration of the detection reliability aswell as the recognition reliability. As a result, a candidate for acharacter which partially appears a character, but entirely appears awrong character string can be removed from characters to be detected.For example, the entire detection reliability R is expressed as follows.

R=E(j·α _(i)+β_(i))  (13)

where α_(i) indicates the detection reliability of adjacent patterns ordetection determined portion, β_(i) indicates the recognitionreliability, and j indicates a weight coefficient.

Then, a character having a larger entire reliability R is selected asthe final character to be detected from the candidates for a pluralityof characters to be detected.

FIG. 68 shows the case where each character is detected from a characterstring ’. In this case, before the character string ‘’ is detected, thediscriminant phase for the statistic and non-statistic processes and thenormal distribution curve of a histogram value are individually obtainedusing learning data.

In the statistic process, parameters c, e, and f shown in FIG. 55 areused as the parameters for use in determining successful or unsuccessfuldetection of a character string. The discriminant phase obtained usingthe learning data is expressed as follows.

0.84×0+0.43×1+0.33×2−145.25=0  (14)

The average value m of the histogram distribution indicating asuccessful detection of learning data shown in FIG. 67 is 128.942. Thestandard deviation is 34.77. The adaptability T is 0.12 according toequation (11). Assuming that the constant of proportionality k is 2, thedistance from the center to the end of the distribution is 77.8according to equation (12).

The average value m of the histogram distribution indicating anunsuccessful detection of learning data shown in FIG. 67 is 71.129. Thestandard deviation is 36.26. The adaptability T is 0.35 according toequation (11). Assuming that the constant of proportionality k is 2, thedistance from the center to the end of the distribution is 92.2according to equation (10).

In FIG. 68, the input pattern of an unknown character is read from aninput image in step S241.

Then, in step S242, a link pattern is extracted using labels, and thelabel numbers <1> through <6> are assigned as shown in FIG. 68 to eachof the extracted link patterns.

In step S245, the detection reliability is quantified based on thestatistic process in step S243 and the non-statistic process in stepS244.

In the statistic process in step S243, the detection reliabilityobtained when adjacent link patterns are integrated is computed based onthe distance h from the discriminant phase to the point in the3-dimensional space having the parameter values c, e, and f. Forexample, this detection reliability a can be expressed as follows.

α=(h−w ₁)/(w ₂ −w ₁)×100  (15)

where

w₁ indicates the leftmost position of the 2-group overlap area

w₂ indicates the rightmost position of the 2-group overlap area

For example, the detection reliability obtained when the patternassigned the label number <1> is integrated into the pattern assignedthe label number <2> is 80. The detection reliability obtained when thepattern assigned the label number <2> is integrated into the patternassigned the label number <3> is 12. The detection reliability obtainedwhen the pattern assigned the label number <3> is integrated into thepattern assigned the label number <4> is 28. The detection reliabilityobtained when the pattern assigned the label number <4> is integratedinto the pattern assigned the label number <5> is 92. The detectionreliability obtained when the pattern assigned the label number <5> isintegrated into the pattern assigned the label number <6> is 5.

In the non-statistic process in step S244, the detection reliability ofthe pattern ‘’ with the superscript strokes is computed based on thedistance h from the discriminant phase to the point in the 3-dimensionalspace having the parameter values p through r in FIG. 60.

For example, the detection reliability obtained when the patternassigned the label number <1> is integrated into thesuperscript-character-pattern of a detection determined portion 271comprising the patterns assigned the label numbers <2> and <3> is 85.

FIG. 65 shows the method of computing the detection reliability in thenon-statistic process in step S244.

In step S251, a pattern 272 is extracted as a candidate for superscriptstrokes. For example, the pattern can be a candidate for superscriptstrokes when there are two adjacent link patterns, and when the ratio ofthe size of the integrated patterns to the average character size of therectangle circumscribing the character string is lower than apredetermined threshold.

In step 252, it is determined whether or not there is a circumscribingrectangle 281 adjacent to the left of the pattern 272 which is acandidate for superscript strokes. If it is determined that there is acircumscribing rectangle 281 adjacent to the left of the pattern 272which is a candidate for superscript strokes, then control is passed tostep S253 and the values of the parameters p through r shown in FIG. 50are output.

In the example shown in FIG. 69, the following equations are expressed.

p=a/b=0.1  (16)

q=b/MX=1.3  (17)

r=(c×d)/(MX×My)²=0.3  (18)

where

a indicates the distance between the right side of the circumscribingrectangle 281 and the left side of the circumscribing rectangle 272;

b indicates the distance between the left side of the circumscribingrectangle 281 and the right side of the circumscribing rectangle 272;

c indicates the area of the circumscribing rectangle 281;

d indicates the area of the circumscribing rectangle 272;

MX indicates an average width of a circumscribing rectangle, and

MY indicates an average height of a circumscribing rectangle.

In step S254, the distance from a discriminant phase 293 to a point inthe 3-dimensional space having the values of the parameters p through ris computed.

To compute the distance from a discriminant phase 293 to a point in the3-dimensional space having the values of the parameters p through r, thediscriminant phase 293 is computed based on the learning pattern. Thediscriminant phase 293 can be obtained by equation (3) based on, forexample, a histogram distribution 292 indicating successful detection ofa character string of a learning pattern and a histogram distribution291 indicating unsuccessful detection of the learning data. The equationfor the discriminant phase 293 using the parameters p through r for usein extracting superscript strokes is expressed by the following equationreferring to a plane in a 3-dimensional space.

0.17×x ₀+0.75×x ₁+0.64×x ₂+30.4=0  (19)

Therefore, the distance h from the discriminant phase 293 is computed bysubstituting the values obtained by equations (16) through (18) forequation (19) as follows.

h=0.17×0.1−0.75×1.3+0.64×0.3+30.4=29.6  (20)

The average value m of the histogram distribution 292 indicating asuccessful detection of learning data is 38. The standard deviation is25. The adaptability T is 0.2 according to equation (11).

The average value m of the histogram distribution 291 indicating anunsuccessful detection of learning data shown is -34. The standarddeviation is 28. The adaptability T is 0.3 according to equation (11).

The left end w₁ of the histogram distribution 292 indicating successfuldetection of learning data is computed as follows by equation (12) withthe constant of proportionality K assumed to be 2.

w₁=38−2×(1+0.2)×25=−22  (21)

The right end w₂ of the histogram distribution 291 indicatingunsuccessful detection of learning data is computed as follows byequation (12) with the constant of proportionality K assumed to be 2.

w₂=−34+2×(1+0.3)×28=38.8  (22)

Therefore, a 2-group overlap area 294 is positioned at the distance of−22 through 38.8 from the discriminant phase.

Next, the detection reliability a is computed in step S255. Thedetection reliability a is computed by substituting the values obtainedby equations (20) through (22) for equation (15) as follows.

α=(29.6−(−22))/(38.8−(−22))×100=85  (23)

Thus, detection determined portion 271 is generated by integrating thepatterns assigned the label numbers <2> and <3>.

In step S246 shown in FIG. 68, the reliability of the statistic andnon-statistic processes is synthesized. At this time, the detectiondetermined portion, if any, is prioritized. As a result, the reliabilityof the detection determined portion 271 is synthesized by priority.

As a result, the detection reliability obtained by integrating thepattern assigned the label number <1> into the pattern of the detectiondetermined portion 271 is 85. The detection reliability obtained byintegrating the pattern of the detection determined portion 271 into thepattern assigned the label number <4> is 30. The detection reliabilityobtained by integrating the pattern assigned the label number <4> intothe pattern assigned the label number <5> is 92. The detectionreliability obtained by integrating the pattern assigned the labelnumber <5> into the pattern assigned the label number <6> is 5.

Patterns are integrated if the detection reliability is higher than apredetermined threshold (for example, 90) or if the detectionreliability is higher than a predetermined threshold (for example, 70)and the ratio of the reliability to the detection reliability of theadjacent detected pattern is higher than a predetermined value (forexample, 5).

Patterns are not integrated if the detection reliability is lower than apredetermined threshold (for example, 8).

For instance, since the detection reliability obtained by integratingthe pattern assigned the label number <1> into the pattern of thedetection determined portion 271 is 85, and the ratio of the reliabilityto the detection reliability of the adjacent pattern assigned the labelnumber <4> is 85/30=2.8, the pattern assigned the label number <1> isnot integrated into the pattern of the detection determined portion 271.The detection reliability obtained by integrating the pattern of thedetection determined portion 271 into the pattern assigned the labelnumber <4> is 30, and therefore, the pattern of the detection determinedportion 271 is not integrated into the pattern assigned the label number<4>.

Since the detection reliability obtained by integrating the patternassigned the label number <4> into the pattern assigned the label number<5> is 92, the pattern assigned the label number <4> is integrated intothe pattern assigned the label number <5>. Since the detectionreliability obtained by integrating the pattern assigned the labelnumber <5> into the pattern assigned the label number <6> is 5, thepattern assigned the label number <5> is not integrated into the patternassigned the label number <6>.

Thus, a circumscribing rectangle 275 corresponding to a detectiondetermined portion 273 obtained by integrating the pattern assigned thelabel number <4> into the pattern assigned the label number <5> and acircumscribing rectangle 276 corresponding to the pattern assigned thelabel number <6> are generated.

Computed then is the detection reliability obtained when the pattern ofthe newly generated detection determined portion 273 is integrated intothe pattern of the detection determined portion 271. The detectionreliability is 60 in the example shown in FIG. 68.

In step 247, a detection candidate 1 and a detection candidate 2 areextracted when the patterns are completely integrated based on thedetection reliability. Then, a recognizing process is performed on eachcharacter of the detection candidates 1 and 2. The detection reliabilityα and β of a character in the detection candidates 1 and 2 is obtainedfor each character, and the sum of the detection reliability α and β isdefined as the entire reliability R.

For example, if the circumscribing rectangles 275, 276, and 278 aredetected as a detection candidate 1, then the recognition reliability βobtained when α character recognizing process is performed on thepattern in a circumscribing rectangle 278 is 80, the recognitionreliability β obtained when a character recognizing process is performedon the pattern in a circumscribing rectangle 275 is 90, the recognitionreliability β obtained when a character recognizing process is performedon the pattern in a circumscribing rectangle 276 is 85.

Since the detection reliability α obtained when the pattern assigned thelabel number <1> is integrated into the pattern of the detectiondetermined portion 271 is 85, the entire reliability R is 345 byequation (13) with the weight coefficient j assumed to be 1.

For example, if the circumscribing rectangles 276, 281, and 282 aredetected as a detection candidate 2, then the recognition reliability βobtained when a character recognizing process is performed on thepattern in a circumscribing rectangle 281 is 83, the recognitionreliability β obtained when a character recognizing process is performedon the pattern in a circumscribing rectangle 282 is 55, the recognitionreliability β obtained when a character recognizing process is performedon the pattern in a circumscribing rectangle 276 is 85.

Since the detection reliability α is 60 when the pattern of thedetection determined portion 271 is integrated into the pattern of thedetection determined portion 273, the entire reliability R is 283.

In step S248, the detection candidate 1 or the detection candidate 2,whichever is larger in entire reliability R, is selected as a candidatefor a successfully detected character. As a result, each of thecharacters ‘’ and ‘’ can be correctly detected in the character string‘’.

Described below is the operations of the obscure character recognizingunit 19 in FIG. 17.

FIG. 70 is a block diagram showing an embodiment of the configuration ofthe obscure character recognizing unit 19.

In FIG. 70, a feature extracting unit 301 extracts a feature of acharacter from an obscure character and represent the extracted featureby a feature vector. An obscure-character dictionary 302 stores afeature vector of each category of obscure characters. A collating unit303 collates the feature vector of a character pattern extracted by thefeature extracting unit 301 with the feature vector of each categorystored in the obscure-character dictionary 302, and computes thedistance D_(ij) (i indicates a feature vector of an unknown character,and j indicates a feature vector of a category in the obscure-characterdictionary 302) between the feature vectors in a feature space. As aresult, the category j indicating the shortest distance D_(ij) betweenthe feature vectors is recognized as an unknown character i.

The distance D_(ij) between the feature vectors in the feature space canbe computed using, for example, an Euclidean distance Σ(i−j)², cityblock distance Σ|i−j|, an identification function such as a discriminantfunction, etc.

Using the distance D_(ij1) from the first category and the distanceD_(ij2) from the second category, a table 1 relating to the firstcategory j1, the second category j2, the distance between categories(D_(ij2)−D_(ij1)), and the reliability is preliminarily generated.Similarly, a table 2 relating to the distance D_(ij1) from the firstcategory, the first category j1, and the reliability is preliminarilygenerated. The data having lower reliability in the tables 1 and 2 isstored in the intermediate process result table.

The deformed character recognizing unit 21 shown in FIG. 7 can bedesigned similarly to the obscure character recognizing unit 19 exceptthat the deformed-character recognizing unit 21 uses adeformed-character dictionary storing feature vectors in each categoryof deformed characters.

Described below is an embodiment of the deletion line recognizing unit26 shown in FIG. 7. The deletion line recognizing unit 26 generates, forexample, a histogram containing sums of the numbers of picture elementsin the horizontal direction for the candidate for a corrected characterextracted by the correction analysis in step S4 in FIG. 8, and removesthe horizontal lines in the area by recognizing that the horizontallines exist in the areas where the histogram value exceeds apredetermined value.

Then, a character is recognized by completing an obscure portion withthe horizontal lines removed and then collating the completed patternwith the dictionary. As a result, if a pattern is recognized as acharacter, then a candidate for a corrected character is regarded as acharacter with deletion lines. If a pattern is rejected, then acandidate for a corrected character is regarded as a normal character.

For example, in FIG. 71, a character ‘5’, which is a candidate for acorrected character, is input as being corrected with double horizontallines. The input pattern is recognized as a corrected character afterdetecting double horizontal lines which indicate the horizontalhistogram value equal to or larger than the threshold N and recognizingthe completed pattern as the category of ‘5’ after the double horizontallines are removed. Assume that a character ‘5’ is input as a candidatefor a corrected character, and horizontal line indicating the horizontalhistogram value equal to or larger than the threshold N is detected. Ifthe horizontal line is removed from the character ‘5’ and the pattern isrejected, then input pattern is not recognized as a corrected character.

Described below is an embodiment of the unique-character analyzing unit23 shown in FIG. 7. The unique-character analyzing unit 23 clustershandwritten characters recognized as belonging to the same category intoa predetermined number of clusters. When clusters belonging to differentcategories indicates a short distance from each other, the charactercategory of the cluster containing a smaller number of elements isamended to the character category of the cluster containing a largernumber of elements, thereby a handwritten character which has beenmisread for a wrong character category can be correctly read.

FIG. 72 shows the clustering process using a feature vector of ahandwritten character recognized as belonging to the character categoryof ‘4’.

FIG. 72 shows the handwritten characters which has been determined asbelonging to the recognition result category of ‘4’ because the patternindicates a shorter distance from the feature vector of the charactercategory of ‘4’ stored in the recognizing dictionary. In thisrecognizing process, the handwritten character ‘2’ is mis-recognized asbelonging to the recognition result category of ‘4’.

In the first clustering process, the handwritten character determined tobelong the character category of ‘4’ is individually regarded as acluster. In the second clustering process, the distance of the featurevectors between the handwritten characters regarded as clusters iscomputed, and the characters indicating a short distance from each otherin feature vector are integrated into one cluster. As a result, thenumber of the clusters is decreased by 1 from 11 to 10 as in the exampleshown in FIG. 72.

In the third and subsequent clustering processes, the number of clusterscan be reduced by computing the distance of feature vectors betweenclusters and integrating the closest feature vectors, resulting in onecluster in the eleventh clustering process.

When clusters are integrated, those individually containing only oneelement are compared in distance with each other using, for example, acity block distance. The center-of-gravity method is used when clusterscontain plural elements. In the center-of-gravity method, when thefeature vector xi of the i-th element (i=1, 2, 3, . . . , M) of thecluster containing M elements is expressed by x_(i)=(x_(i1), x_(i2),x_(i3), . . . , x_(iN)), the representative vector x_(m) of the clustersis represented by an average of the feature vector x_(i) of the elementsof the clusters as follows. $\begin{matrix}{{Xm} = {\frac{1}{M}{\sum\limits_{i = 1}^{M}X_{i}}}} & (24)\end{matrix}$

Clusters containing plural elements are compared with each other bycomputing the city block distance between the representative vectorsx_(m).

If the clustering processes are repeated until the number of clusters isreduced to 1, the handwritten character ‘2’ mis-recognized for belongingto the character category of ‘4’ is regarded as belonging to the samecharacter category as the handwritten character ‘4’ correctly recognizedas belonging to the character category of ‘4’. Therefore, a clusteringaborting condition is set to abort the clustering processes.

The clustering aborting condition can be satisfied when

(1) the number of clusters reaches a predetermined value (for example,3);

(2) the distance between clusters exceeds a predetermined threshold whenthe clusters are integrated; and

(3) the increase ratio of the distance between clusters exceeds apredetermined threshold when the clusters are integrated.

FIG. 73 is a flowchart showing the clustering process;

As shown in FIG. 73, only the feature vector of the handwrittencharacter recognized as belonging to a specific character category isextracted in step S261. Each of the extracted feature vectors of thehandwritten characters is regarded as one cluster.

In step S262, the clustering aborting condition is set to abort theclustering processes.

In step S263, two clusters closest to each other in all clusters areselected about a specific character.

It is determined in step S264 whether or not the cluster abortingconditions set in step S262 are satisfied. If the cluster abortingconditions set in step S262 are not satisfied, then control is passed tostep S265, the two clusters selected in step S263 are integrated,control is returned to step S263, and clusters are repeatedlyintegrated.

If it is determined in step S264 that the clustering aborting conditionsare satisfied after repeating the cluster integrating processes, thencontrol is passed to step S266, and it is determined whether or not theclustering processes have been performed on all character categories. Ifall clustering processes have not been performed on all charactercategories, control is returned to step 261, and the clustering processis performed on the unprocessed character categories.

If it is determined in step S266 that the clustering processes have beenperformed on all character categories, then control is passed to stepS267, and the clustering results are stored in the memory.

Next, a handwritten character mis-recognized for belonging to adifferent category is correctly read based on the clustering resultobtained in the clustering process.

FIG. 74 shows the process of correctly reading a handwritten character‘2’, which is mis-recognized for belonging to the character category of‘4’, as the character category ‘2’.

FIG. 74 shows the handwritten character determined to belong to therecognition result category of ‘2’, and the handwritten characterdetermined to belong to the recognition result category of ‘4’. Thehandwritten character ‘3’ is mis-recognized for belonging to therecognition result category of ‘2’, and the handwritten character ‘2’ ismis-recognized for belonging to the recognition result category of ‘4’.The handwritten character ‘4’ is rejected for not belonging to anyrecognition result category.

Performing the clustering process by setting the clustering abortingcondition when the number of clusters in a specific category reaches 3allows clusters a, b, and c to be generated for the recognition resultcategory of ‘2’ and clusters d, e, and f to be generated for therecognition result category of ‘4’. Clusters g, h, and i are generatedfor the three rejected handwritten characters ‘4’.

Then, clusters a, b, and c belonging to the recognition result categoryof ‘2’ or clusters d, e, and f belonging to the recognition resultcategory of ‘4’ whichever contains a smaller number of characters isextracted as a candidate for a mis-read cluster.

The distances from mis-read candidate cluster a to each of otherclusters b, c, d, e, and f, and the distances from mis-read candidatecluster d to each of other clusters a, b, c, e, and f are computed.Cluster b is extracted as the cluster closest to mis-read candidatecluster a. It is determined whether or not the distance between mis-readcandidate cluster a and cluster b is shorter than a predetermined value.Since the distance between mis-read candidate cluster a and cluster b isnot shorter than a predetermined value, mis-read candidate cluster a isrejected.

As a result, the handwritten character ‘3’ mis-recognized for belongingto the recognition result category of ‘2’ is removed from therecognition result category of 2.

Cluster b is extracted as the cluster closest to mis-read candidatecluster d. It is determined whether or not the distance between mis-readcandidate cluster d and cluster b is shorter than a predetermined value.Since the distance between mis-read candidate cluster d and cluster b isshorter than a predetermined value, mis-read candidate cluster d isintegrated into cluster b to generate cluster j. It is determined thatcluster j belongs to the recognition result category of ‘2’ to whichcluster b containing the larger number of elements belongs. Thus, thehandwritten character ‘2’ which has been mis-read for ‘4’ and determinedto belong to mis-read candidate cluster d can be correctly read.

Next, the distances between clusters g, h, and i rejected as notbelonging to any recognition result category and other clusters athrough f are computed. Cluster e is extracted as the cluster closest tocluster g. It is determined whether or not the distance between clusterg and cluster e is shorter than a predetermined value. Since thedistance between cluster g and cluster e is shorter than a predeterminedvalue, cluster g is integrated into cluster e.

Cluster e is extracted as the cluster closest to cluster h. It isdetermined whether or not the distance between cluster h and cluster eis shorter than a predetermined value. Since the distance betweencluster h and cluster e is shorter than a predetermined value, cluster his integrated into cluster e. After integrating clusters g and h intocluster e, cluster k is generated. It is determined that cluster kbelongs to the recognition result category of ‘4’ to which cluster econtaining the larger number of elements belongs. Therefore, thehandwritten character ‘4’ which has been rejected as beingunrecognizable can be correctly read.

Cluster e is extracted as the cluster closest to cluster i. It isdetermined whether or not the distance between cluster i and cluster eis shorter than a predetermined value. Since the distance betweencluster i and cluster e is not shorter than a predetermined value,cluster i is not integrated into cluster e.

FIG. 75 is a flowchart showing the character category recognition resultamending process.

In FIG. 75, the data of the clustering result obtained in the clusteringprocess shown in FIG. 73 is read from the memory in step S271.

Then, in step S272, the distance between clusters is computed andcompared for all clusters in all categories obtained in the clusteringprocess shown in FIG. 73.

In step S273, it is determined whether or not the distance betweenclusters is shorter than a predetermined threshold. If the distancebetween any clusters is shorter than a predetermined threshold, thencontrol is passed to step S274, and the clusters are integrated. If thedistance between any clusters is not shorter than a predeterminedthreshold, the clusters are rejected.

Assume that the threshold of the distance between the clusters in theintegration of the clusters is a multiple of the constant of thedistance between the vectors in one of, for example, two clusterswhichever contains a larger number of elements. That is, when cluster Acontaining m elements is integrated into cluster B containing N (M>N)elements, the distance d_(t)h between the vectors in cluster A isexpressed as follows. $\begin{matrix}{{d_{t}h} = {\frac{1}{M}{\sum\limits_{i = 1}^{M - 1}{\underset{j \neq 1}{\sum\limits_{j = {i + 1}}^{M}}{{{xa}_{i} - {xa}_{j}}}}}}} & (25)\end{matrix}$

where xai (i=1, 2, . . . , M) indicates the feature vector in cluster A.

Therefore, the condition of integrating clusters can be represented asfollows with the constant set to 1.5.

|xam−xbm|<1.5d _(t)h

where xam indicates the representative vector of cluster A, xbmindicates the representative vector of cluster B.

Then, in step S275, the character categories are determined in allclusters integrated in step S274.

In step S276, it is determined whether or not the character categoriesof the integrated clusters are different from each other. If thecharacter categories of the integrated clusters are different from eachother, then control is passed to step S277, and the character categorycontaining a smaller number of elements is amended to the charactercategory of the cluster containing a larger number of elements. Then,control is passed to step S278. If the character categories of theclusters match, then step S277 is skipped and control is passed to stepS278.

Then, in step S278, a character category is output for a character inthe cluster.

The operations of the pattern recognizing apparatus according to thepresent invention is practically described by referring to the casewhere the form shown in FIG. 76 is processed.

FIG. 76 shows an example of the form input to the pattern recognizingapparatus according to an embodiment of the present invention.

The form shown in FIG. 76 contains a free-pitch box having the boxnumber 1; a one-character box having the box number 2, 3, or 4; a blockcharacter box having the box number 5; and an irregular table having thebox number 6. The free-pitch box having the box number 1 contains thebox-touching character ‘5’ corrected with double horizon lines; thebox-touching characters ‘3’ and ‘2’; the box-touching obscure character‘7’; the unique characters ‘4’ and ‘6’; and the partly-outside-boxunique character ‘4’.

The one-character box having the box number 2 contains ‘5’- Theone-character box having the box number 3 contains ‘3’. Theone-character box having the box number 4 contains thepartly-outside-box character ‘8’ corrected with double horizontal lines.In the block character boxes having the box number 5, the character boxhaving the box number 5-1 contains the unique character ‘6’ correctedwith double horizontal lines, the character box having the box number5-2 contains the box-touching character ‘2’, and the character boxhaving the box number 5-3 contains the unique character ‘4’.

In the irregular tables having the box number 6, the character boxhaving the box number 6-1-1 contains the partly-outside-box characters131, ‘2’, and ‘1’, the character box having the box number 6-1-2contains the character 6, 3, and 8, and the character boxes having thebox numbers 6-1-3, 6-1-4-1, 6-1-4-2, 6-1-4-3, 6-2-1, 6-2-2, and 6-2-3are kept blank. The entire irregular table having the character boxnumber 6 is corrected with the mark ‘x’.

Next, the environment recognizing system 11 shown in FIG. 7 performs theprocess shown in FIGS. 9 through 12, thereby extracting the state of aninput image from the form shown in FIG. 76.

For example, the free-pitch character box having the box number 1,one-character boxes having the box numbers 2, 3, and 4, the blockcharacter box having the box number 5, and the irregular table havingthe box number 6 are extracted from the form shown in FIG. 76 byperforming the layout analysis shown in FIG. 10. Additionally, eightpatterns are extracted as candidates for characters from the free-pitchbox having the box number 1. A pattern is extracted as a candidate for acharacter from each of the one-character boxes having the box numbers 2,3, and 4. Three patterns are extracted as candidates for characters fromthe block character box having the box number 5. Three patterns areextracted as candidates for characters from the character box having thebox number 6-1-1. Three patterns are extracted as candidates forcharacters from the character box having the box number 6-1-2. Nopatterns are extracted as candidates for characters from the characterboxes having the box numbers 6-1-3, 6-1-4-1, 6-1-4-2, 6-1-4-3, 6-2-1,6-2-2, and 6-2-3.

To extract a character string from the form shown in FIG. 76, the textextracting method shown in FIGS. 18 and 19 is used. To extract a ruledline from the form shown in FIG. 76, the ruled line extracting methodshown in FIGS. 20 and 26 is used. To extract a character box or a tablefrom the form shown in FIG. 76, the box extracting method shown in FIGS.27 and 28 is used.

The first, second, fifth, and eighth patterns extracted from thefree-pitch box having the box number 1 are regarded as candidates forbox-touching characters. The pattern extracted from the one-characterbox having the box number 4, the pattern extracted from the characterbox having the box number 5-2, the first pattern extracted from thecharacter box having the box number 6-1-1 are also regarded ascandidates for box-touching characters.

To extract a candidate for a box-touching character from the form shownin FIG. 76, the box-touching character extracting method shown in FIGS.31 and 32 is used.

Through the quality analysis shown in FIG. 11, obscure, deformed, andhigh-quality characters are detected in the form shown in FIG. 76. Inthis example, the quality of images is normal, but obscure, deformed, orhigh-quality characters are not detected.

A candidate for a corrected character is extracted from the list shownin FIG. 76 through the correction analysis shown in FIG. 12. In thisexample, the first pattern extracted from the free-pitch box having thebox number 1, the patterns extracted from the one-character boxes havingthe box numbers 2 and 4, the pattern extracted from the character boxhaving the box number 5-1, and the pattern extracted from the irregulartable having the box number 6 are regarded as candidates for correctedcharacters.

To extract a candidate for a corrected character from the form shown inFIG. 76, the feature extracting method shown in FIG. 34 is used, forexample.

Next, the environment recognizing system 11 generates an intermediateprocess result table containing the state extracted from the form in theprocesses shown in FIGS. 9 through 12 for each of the candidates for thecharacters extracted from the input image.

FIG. 77 shows the intermediate process result table containing the stateextracted from the form in the processes in FIGS. 9 through 12.

In FIG. 77, the column of the character box having the box number 1contains ‘free-pitch’ for the type of box and ‘8’ for the number ofcharacters. The column for the first pattern in the box having the boxnumber 1 contains ‘YES’ indicating the existence of a box-touchingcharacter, ‘YES 2’ indicating the existence of deletion lines, and‘NORMAL’ indicating the quality. The column for the second pattern inthe box having the box number 1 contains ‘YES’ indicating the existenceof a box-touching character, ‘NO’ indicating the non-existence ofdeletion lines, and ‘NORMAL’ indicating the quality. The column for theeighth pattern in the box having the box number 1 contains ‘YES’indicating the existence of a box-touching character, ‘NO’ indicatingthe non-existence of deletion lines, and ‘NORMAL’ indicating thequality.

‘YES 1’ indicating the existence of the deletion lines refers to theexistence of a candidate for deletion lines for a plurality ofcharacters. ‘YES 2’ indicating the existence of the deletion linesrefers to the existence of a candidate for deletion lines for a singlecharacter.

The column of the character box having the box number 2 contains‘one-character’ for the type of box, ‘NO’ indicating the non-existenceof a box-touching character, ‘YES 2’ indicating the existence ofdeletion lines, ‘NORMAL’ indicating the quality, and ‘1’ indicating thenumber of characters. The column of the character box having the boxnumber 3 contains ‘one-character’ for the type of box, ‘NO’ indicatingthe non-existence of a box-touching character, ‘NO’ indicating thenon-existence of deletion lines, ‘NORMAL’ indicating the quality, and‘1’ indicating the number of characters. The column of the character boxhaving the box number 4 contains ‘one-character’ for the type of box,‘YES’ indicating the existence of a box-touching character, ‘YES 2’indicating the existence of deletion lines, ‘NORMAL’ indicating thequality, and ‘1’ indicating the number of characters.

The column of the character box having the box number 5 contains‘INDIVIDUAL CHARACTER BOXES’ indicating the type of box, ‘3’ indicatingthe number of characters. The column of the character box having the boxnumber 5-1 contains ‘NO’ indicating the non-existence of a box-touchingcharacter, ‘YES 2’ indicating the existence of deletion lines, ‘NORMAL’indicating the quality, and ‘1’ indicating the number of characters. Thecolumn of the character box having the box number 5-2 contains ‘YES’indicating the existence of a box-touching character, ‘NO’ indicatingthe non-existence of deletion lines, ‘NORMAL’ indicating the quality,and ‘1’ indicating the number of characters.

The column of the character box having the box number 6 contains ‘TABLE’indicating the type of character box. The column of the character boxhaving the box number 6-1-1 contains ‘FREE-PITCH’ indicating the type ofcharacter box, ‘YES’ indicating the existence of a box-touchingcharacter, ‘YES 1’ indicating the existence of deletion lines, and‘NORMAL’ indicating the quality. The column of the character box havingthe box number 6-2-2 contains ‘FREE-PITCH’ indicating the type ofcharacter box, ‘NO’ indicating the non-existence of a box-touchingcharacter, ‘YES 1’ indicating the existence of deletion lines, and‘NORMAL’ indicating the quality. The environment recognizing system 11performs the process shown in FIG. 13 based on the state extracted fromthe form in the process shown in FIGS. 9 through 12.

That is, the environment recognizing system 11 determines which processis to be called by referring to the process order control rules, theprocess performed by the basic character recognizing unit 17, characterstring recognizing unit 15, box-touching character recognizing unit 13,obscure character recognizing unit 19, or deformed character recognizingunit 21 of the character recognizing unit 12 shown in FIG. 7, or theprocess performed by the deletion line recognizing unit 26 or noiserecognizing unit 28 of the non-character recognizing unit 25. Thedetermined process is entered in the column ‘CALLING PROCESS’ of theintermediate process result table shown in FIG. 77. Then, it determines,by referring to the process order table, in what order the processentered in the column ‘CALLING PROCESS’ of the intermediate processresult table shown in FIG. 77 should be performed. The determined orderis entered in the column ‘PROCESS ORDER’ of the intermediate processresult table shown in FIG. 77.

The process order control rule can be specified as follows.

(A1) If a column indicating the state of the intermediate process resulttable contains ‘YES’ for a specific process object and the processcorresponding to the state has not been performed, then the processcorresponding to the state is entered in the column ‘CALLING PROCESS’ ofthe intermediate process result table.

(A2) If all columns indicating the states of the intermediate processresult table contains ‘NO’ or ‘NORMAL’ for a specific process object andthe process to be performed by the basic character recognizing unit 17has not been performed, then “BASIC” is entered in the column ‘CALLINGPROCESS’ of the intermediate process result table.

(A3) If there are a plurality of processes corresponding to the stateentered in the intermediate process result table for a process object,then the process order table for use in determining the order of aplurality of processes is accessed to rearrange the order of the‘CALLING PROCESS’.

(A4) If a process corresponding to the state entered in the intermediateprocess result table has been performed on a process object, then thecompleted process is entered in the column ‘COMPLETION OF PROCESS’,information about the suspension or completion of the next instructionor process is entered in the column ‘PROCESS INSTRUCTION’ of theintermediate process result table, and the order in the column ‘CALLINGPROCESS’ of the intermediate process result table is rearrangedaccording to the information.

FIG. 78 shows an example of the process order table.

In FIG. 78, the process order table stores the following procedures.

(B1) If only one process is entered in the column ‘CALLING PROCESS’ ofthe intermediate process result table for a process object, then theprocess is entered in the column ‘PROCESS ORDER’ of the intermediateprocess result table.

(B2) If ‘BLACK-CHARACTER-BOX/FREE-PITCH’ is entered in the column‘CALLING PROCESS’ of the intermediate process result table for a processobject, then ‘BLACK-CHARACTER-BOX FREE-PITCH’ is entered in the column‘PROCESS ORDER’ of the intermediate process result table.

(B3) If ‘DELETION LINES (YES 2)/BLACK-CHARACTER-BOX’ is entered in thecolumn ‘CALLING PROCESS’ of the intermediate process result table for aprocess object, then ‘BLACK-CHARACTER-BOX→CHARACTER DELETION LINES’ isentered.

(B4) If ‘BLACK-CHARACTER-BOX/FREE-PITCH/DELETION (YES 2)’ is entered inthe column ‘CALLING PROCESS’ of the intermediate process result tablefor a process object, then ‘BLACK-CHARACTER-BOX→CHARACTER DELETIONLINES→FREE-PITCH’ is entered in the column ‘PROCESS ORDER’ of theintermediate process result table.

(B5) If ‘BLACK-CHARACTER-BOX/FREE-PITCH/DELETION (YES 1)’ is entered inthe column ‘CALLING PROCESS’ of the intermediate process result tablefor a process object, then ‘DELETIONLINES→BLACK-CHARACTER-BOX→FREE-PITCH’ for a plurality of characters isentered in the column ‘PROCESS ORDER’ of the intermediate process resulttable.

(B6) If ‘FREE-PITCH/DELETION (YES 1)’ is entered in the column ‘CALLINGPROCESS’ of the intermediate process result table for a process object,then ‘PLURAL-CHARACTER DELETION LINES→FREE-PITCH’ is entered in thecolumn ‘PROCESS ORDER’ of the intermediate process result table.

(B7) If ‘PROCESSES A, B, AND C’ is entered in the column ‘CALLINGPROCESS’ of the intermediate process result table for a process object,‘PROCESS B→PROCESS A→PROCESS C’ is entered in the column ‘PROCESS ORDER’of the intermediate process result table, and ‘PROCESS B’ is entered inthe column ‘COMPLETION OF PROCESS’ of the intermediate process resulttable, then the column ‘PROCESS ORDER’ of the intermediate processresult table is updated into ‘PROCESS A→PROCESS C’.

(B8) If ‘PROCESSES A, B, AND C’ is entered in the column ‘CALLINGPROCESS’ of the intermediate process result table for a process object,‘PROCESS B→PROCESS A→PROCESS C’ is entered in the column ‘PROCESS ORDER’of the intermediate process result table, ‘PROCESS B’ is entered in thecolumn ‘COMPLETION OF PROCESS’ of the intermediate process result table,and ‘SKIPPING TO PROCESS C’ is entered in the column ‘PROCESSINSTRUCTION’ of the intermediate process result table, then the column‘PROCESS ORDER’ of the intermediate process result table is updated into‘PROCESS C’.

(B9) If ‘PROCESSES A, B, AND C’ is entered in the column ‘CALLINGPROCESS’ of the intermediate process result table for a process object,‘PROCESS B→PROCESS A→PROCESS C’ is entered in the column ‘PROCESS ORDER’of the intermediate process result table, ‘PROCESS B’ is entered in thecolumn ‘COMPLETION OF PROCESS’ of the intermediate process result table,and ‘INVERTING ORDER BETWEEN PROCESSES C AND A’ is entered in the column‘PROCESS INSTRUCTION’ of the intermediate process result table, then thecolumn ‘PROCESS ORDER’ of the intermediate process result table isupdated into ‘PROCESS C→PROCESS A’.

(B10) If ‘PROCESSES B, AND A’ is entered in the column ‘CALLING PROCESS’of the intermediate process result table for a process object, ‘PROCESSA’ is entered in the column ‘COMPLETION OF PROCESS’ of the intermediateprocess result table, and ‘TERMINATION’ is entered in the column‘PROCESS INSTRUCTION’ of the intermediate process result table, then‘TERMINATION’ is entered in the column ‘PROCESS ORDER’ of theintermediate process result table.

FIG. 79 shows an example of entering in the column ‘CALLING PROCESS’ theprocess to be called based on the state of an input image entered in theintermediate process result table shown in FIG. 77, and of entering inthe column ‘PROCESS ORDER’ the order of performing the process enteredin the column ‘CALLING PROCESS’.

In FIG. 79, the column of the character box having the box number 1contains ‘FREE-PITCH’ for the type of box. The column for the firstpattern in the box having the box number 1 contains ‘YES’ indicating theexistence of a box-touching character, and ‘YES 2’ indicating theexistence of deletion lines. Therefore, according to (A1) in the processorder control rule, ‘BLACK-CHARACTER-BOX/FREE-PITCH/DELETION LINE (YES2)’ is entered in the column ‘CALLING PROCESS’, (B4) of the processorder table is referred to according to (A3) of the process ordercontrol rule, and ‘BLACK-CHARACTER-BOX→ONE-CHARACTER DELETIONLINE→FREE-PITCH’ is entered in the column ‘PROCESS ORDER’.

The column for the eighth pattern in the box having the box number 1contains ‘YES’ indicating the existence of a box-touching character,‘NO’ indicating the non-existence of deletion lines, and ‘NORMAL’ as‘QUALITY’. Therefore, according to (A1) in the process order controlrule, ‘BLACK-CHARACTER-BOX/FREE-PITCH’ is entered in the column ‘CALLINGPROCESS’, (B2) of the process order table is referred to according to(A3) of the process order control rule, and‘BLACK-CHARACTER-BOX→FREE-PITCH’ is entered in the column ‘PROCESSORDER’.

The column for the second pattern in the box having the box number 1contains ‘YES’ indicating the existence of a box-touching character,‘NO’ indicating the non-existence of deletion lines, and ‘NORMAL’ as‘QUALITY’. Therefore, according to (A1) in the process order controlrule, ‘BLACK-CHARACTER-BOX/FREE-PITCH’ is entered in the column ‘CALLINGPROCESS’, (B2) of the process order table is referred to according to(A3) of the process order control rule, and‘BLACK-CHARACTER-BOX→FREE-PITCH’ is entered in the column ‘PROCESSORDER’.

The column of the character box having the box number 2 contains ‘ONECHARACTER’ for the type of box, ‘NO’ indicating the non-existence of abox-touching character, ‘YES 2’ indicating the existence of deletionlines, and ‘NORMAL’ indicating the quality. Therefore, according to (A1)in the process order control rule, ‘DELETION LINE (YES 2)’ is entered inthe column ‘CALLING PROCESS’, and ‘ONE-CHARACTER DELETION LINE’ isentered in the column ‘PROCESS ORDER’ according to (A1) in the processorder control rule.

The column of the character box having the box number 3 contains ‘ONECHARACTER’ for the type of box, ‘NO’ indicating the non-existence of abox-touching character, ‘NO’ indicating the non-existence of deletionlines, and ‘NORMAL’ indicating the quality. Therefore, according to (A2)in the process order control rule, ‘BASIC’ is entered in the column‘CALLING PROCESS’, and ‘BASIC’ is entered in the column ‘PROCESS ORDER’according to (A1) in the process order control rule.

The column of the character box having the box number 4 contains ‘ONECHARACTER’ for the type of box, ‘YES’ indicating the existence of abox-touching character, ‘YES 2’ indicating the existence of deletionlines, and ‘NORMAL’ indicating the quality. Therefore, according to (A1)in the process order control rule, ‘BLACK-CHARACTER-BOX/DELETION LINE(YES 2)’ is entered in the column ‘CALLING PROCESS’, (B3) of the processorder table is referred to and ‘BLACK-CHARACTER-BOX→ONE-CHARACTERDELETION LINE’ is entered in the column ‘PROCESS ORDER’ according to(A3) in the process order control rule.

The column of the character box having the box number 5 contains‘INDIVIDUAL CHARACTER BOXES’ for the type of box. The column of thecharacter box having the box number 5-1 contains ‘NO’ indicating thenon-existence of a box-touching character, ‘YES 2’ indicating theexistence of deletion lines, and ‘NORMAL’ indicating the qualityTherefore, according to (A1) in the process order control rule,‘DELETION LINE (YES 2)’ is entered in the column ‘CALLING PROCESS’, and‘ONE-CHARACTER DELETION LINE’ is entered in the column ‘PROCESS ORDER’according to (A1) of the process order control rule.

The column of the character box having the box number 5-2 contains ‘YES’indicating the existence of a box-touching character, ‘NO’ indicatingthe non-existence of deletion lines, and ‘NORMAL’ indicating thequality. Therefore, according to (A1) in the process order control rule,‘BLACK-CHARACTER-BOX’ is entered in the column ‘CALLING PROCESS’, and‘BLACK-CHARACTER-BOX’ is entered in the column ‘PROCESS ORDER’ accordingto (A1) of the process order control rule.

The column of the character box having the box number 5-3 contains ‘NO’indicating the non-existence of a box-touching character, ‘NO’indicating the non-existence of deletion lines, and ‘NORMAL’ indicatingthe quality. Therefore, according to (A2) in the process order controlrule, ‘BASIC’ is entered in the column ‘CALLING PROCESS’, and ‘BASIC’ isentered in the column ‘PROCESS ORDER’ according to (A1) in the processorder control rule.

The column of the character box having the box number 6 contains ‘TABLE’indicating the type of character box. The column of the character boxhaving the box number 6-1-1 contains ‘FREE-PITCH’ indicating the type ofcharacter box, ‘YES’ indicating the existence of a box-touchingcharacter, ‘YES 1’ indicating the existence of deletion lines, and‘NORMAL’ indicating the quality. Therefore, according to (A1) in theprocess order control rule, ‘BLACK-CHARACTER-BOX/FREE-PITCH/DELETIONLINE (YES 1)’ is entered in the column ‘CALLING PROCESS’, (B5) of theprocess order table is referred to according to (A3) of the processorder control rule, and ‘PLURAL-CHARACTER DELETIONLINE→BLACK-CHARACTER-BOX→FREE-PITCH’ is entered in the column ‘PROCESSORDER’.

The column of the character box having the box number 6-2-2 contains‘FREE-PITCH’ indicating the type of character box, ‘NO’ indicating thenon-existence of a box-touching character, ‘YES 1’ indicating theexistence of deletion lines, and ‘NORMAL’ indicating the quality.Therefore, according to (A1) in the process order control rule,‘FREE-PITCH/DELETION LINE (YES 1)’ is entered in the column ‘CALLINGPROCESS’, (B6) of the process order table is referred to according to(A3) of the process order control rule, and ‘PLURAL-CHARACTER DELETIONLINE→FREE-PITCH’ is entered in the column ‘PROCESS ORDER’.

Next, the first recognizing process shown in FIG. 78 is performed byreferring to the process execution rule based on the intermediateprocess result table shown in FIG. 79 with the data entered in thecolumns ‘CALLING PROCESS’ and ‘PROCESS ORDER’. The completed recognizingprocess is entered in the column ‘COMPLETION OF PROCESS’ on theintermediate process result table, and the reliability obtained in therecognizing process is entered in the column ‘RELIABILITY’ on theintermediate process result table.

The column ‘PROCESS ORDER’ on the intermediate process result table isupdated according to (B7) through (B9) on the process order table shownin FIG. 78. If the next process is specified according to the processexecution rule, the process is entered in the column ‘PROCESSINSTRUCTION’ on the intermediate process result table.

The process execution rule can be described as follows.

(C1) If there is a process entered in the column ‘PROCESS ORDER’ on theintermediate process result table for a process object, then a processassigned the highest priority is performs. If the performed process hasbeen completed, the completed process is entered in the column‘COMPLETION OF PROCESS’ on the intermediate process result table, andthe process is deleted from the column ‘PROCESS ORDER’ on theintermediate process result table. If the next process is specified, theprocess is entered in the ‘PROCESS INSTRUCTION’ on the intermediateprocess result table.

(C2) If a process is performed and it is determined that a pattern is acharacter pattern, not a non-character pattern, and the character codeis computed with the reliability at or above a predetermined value, thencalling a character recognizing process through the ‘PERSONALHANDWRITING FEATURES’ is entered in the column ‘PROCESS INSTRUCTION’ onthe intermediate process result table.

(C3) If a process is performed and it is determined that a pattern is adeletion line, an the deletion line is computed with the reliability atand above a predetermined value, then ‘TERMINATION’ is entered in thecolumn ‘PROCESS INSTRUCTION’ on the intermediate process result tableand the subsequent processes entered in the column ‘PROCESS ORDER’ onthe intermediate process result table are aborted and the process isterminated.

(C4) If the column ‘PROCESS ORDER’ on the intermediate process resulttable starts with the entry ‘FREE-PITCH’ and the processes of the samebox number for other process object before the ‘FREE-PITCH’ have notbeen processed, then the column ‘PROCESS ORDER’ of all process objectsof the same box number starts with the entry ‘FREE-PITCH’ and allprocesses ‘FREE-PITCH’ for all process objects of the same box numberare simultaneously performed.

(C5) If all processes entered in the column ‘PROCESS ORDER’ on theintermediate process result table have been completed and ‘TERMINATION’or ‘PERSONAL HANDWRITING FEATURES’ is entered in the column ‘PROCESSINSTRUCTION’ on the intermediate process result table for all processobjects, then the character recognizing process is called and performedthrough the ‘PERSONAL HANDWRITING FEATURES’ on the process object forwhich the ‘PERSONAL HANDWRITING FEATURES’ is entered in the column‘PROCESS INSTRUCTION’. If the character recognizing process has beenperformed through the ‘PERSONAL HANDWRITING FEATURES’, then‘TERMINATION’ is entered in the column ‘PROCESS INSTRUCTION’ on theintermediate process result table.

(C6) If ‘TERMINATION’ is entered in the column ‘PROCESS ORDER’ on theintermediate process result table for all process objects, then allprocesses are terminated and the recognition results are output.

FIG. 80 shows an example of performing a recognizing process byreferring to the process execution rule based on the intermediateprocess result table shown in FIG. 79; entering the reliability obtainedin the recognizing process in the column ‘RELIABILITY’ on theintermediate process result table; updating the column ‘PROCESS ORDER’on the intermediate process result table based on the process executionrule; and entering data in the column ‘PROCESS INSTRUCTION’ on theintermediate process result table.

Since ‘BLACK-CHARACTER-BOX’ is specified for the first pattern in thecolumn ‘PROCESS ORDER’ of the character box having the box number 1 onthe intermediate process result table shown in FIG. 79, the process ofthe box-touching character recognizing unit 13 shown in FIG. 3 isperformed on the first pattern extracted from the free-pitch characterbox having the box number 1 as shown in FIG. 76 corresponding to‘BLACK-CHARACTER-BOX’ according to the process execution rule (C1).

The box-touching character recognizing unit 13 recognizes a character bycompleting or re-completing the character for the pattern from which itscharacter box is removed as shown in FIGS. 43 and 44. If a patterncannot be recognized at acceptable reliability even after the charactercompleting or re-completing process, then the knowledge table 14 isreferred to and a character re-completing process is performed on thelearning character shown in FIGS. 46 through 55 to successfullyrecognize a box-touching character.

If the recognition reliability of the first pattern extracted from thefree-pitch box having the box number 1 shown in FIG. 76 is computed as20% in the character recognizing process performed by the box-touchingcharacter recognizing unit 13, then the first pattern extracted from thefree-pitch box having the box number 1 as shown in FIG. 76 is regardedas a non-character and ‘REJECT’ is entered in the column ‘CHARACTERCODE’ on the intermediate process result table, and ‘20%’ is entered inthe column ‘RELIABILITY’ on the intermediate process result table.‘BLACK-CHARACTER-BOX’ is entered in the column ‘COMPLETION OF PROCESS’on the intermediate process result table, and the column ‘PROCESS ORDER’on the intermediate process result table is updated to ‘ONE-CHARACTERDELETION LINE→FREE-PITCH’.

Next, since ‘BLACK-CHARACTER-BOX’ is specified for the second pattern inthe column ‘PROCESS ORDER’ of the character box having the box number 1on the intermediate process result table shown in FIG. 79, the processof the box-touching character recognizing unit 13 shown in FIG. 7 isperformed on the second pattern extracted from the free-pitch characterbox having the box number 1 as shown in FIG. 76 corresponding to‘BLACK-CHARACTER-BOX’ according to the process execution rule (C1), anda character recognizing process is performed on a box-touchingcharacter.

The second pattern extracted from the free-pitch box having the boxnumber 1 shown in FIG. 76 is recognized as the character category 3 withthe recognition reliability of 60% in the character recognizing processperformed by the box-touching character recognizing unit 13, then ‘3’ isentered in the column ‘CHARACTER CODE’ on the intermediate processresult table, and ‘60%’ is entered in the column ‘RELIABILITY’ on theintermediate process result table. ‘BLACK-CHARACTER-BOX’ is entered inthe column ‘COMPLETION OF PROCESS’ on the intermediate process resulttable, and the column ‘PROCESS ORDER’ on the intermediate process resulttable is updated to ‘FREE-PITCH’.

Next, since ‘BLACK-CHARACTER-BOX’ is specified for the eighth pattern inthe column ‘PROCESS ORDER’ of the character box having the box number 1on the intermediate process result table shown in FIG. 79, the processof the box-touching character recognizing unit 13 shown in FIG. 7 isperformed on the eighth pattern extracted from the free-pitch characterbox having the box number 1 as shown in FIG. 76 corresponding to‘BLACK-CHARACTER-BOX’ according to the process execution rule (C1), anda character recognizing process is performed on a box-touchingcharacter.

The eighth pattern extracted from the free-pitch box having the boxnumber 1 shown in FIG. 76 is recognized as the character category 4 withthe recognition reliability of 95% in the character recognizing processperformed by the box-touching character recognizing unit 13, then ‘4’ isentered in the column ‘CHARACTER CODE’ on the intermediate processresult table, and ‘95%’ is entered in the column ‘RELIABILITY’ on theintermediate process result table.

‘BLACK-CHARACTER-BOX’ is entered in the column ‘COMPLETION OF PROCESS’on the intermediate process result table, and the column ‘PROCESS ORDER’on the intermediate process result table is updated to ‘FREE-PITCH’.

Next, since ‘ONE-CHARACTER DELETION LINE’ is specified in the column‘PROCESS ORDER’ of the character box having the box number 2 on theintermediate process result table shown in FIG. 79, the process of thedeletion line recognizing unit 26 shown in FIG. 7 is performed on thepattern extracted from the one-character box having the box number 2 asshown in FIG. 76 corresponding to ‘ONE-CHARACTER DELETION LINE’according to the process execution rule (C1).

The deletion line recognizing unit 26 removes horizontal linesindicating the histogram value equal to or larger than a predeterminedvalue from a pattern extracted as a candidate for a corrected characteras shown in FIG. 71. If the pattern from which the horizontal lines areremoved is recognized as a character, then the pattern extracted as thecandidate for the corrected character can be recognized as a correctedcharacter by recognizing the removed horizontal lines as deletion lines.If the pattern from which the horizontal lines indicating the histogramvalue equal to or larger than the predetermined value are removed isrejected, then the removed horizontal lines are not regarded as deletionlines but a portion of a character. Thus, the pattern extracted as acandidate for a corrected character is recognized as a normal character.

In the deletion line recognizing process performed by the deletion linerecognizing unit 26, the recognition reliability of the patternextracted from the one-character box having a box number 2 shown in FIG.76 is computed as 10%. The pattern extracted from the one-character boxhaving a box number 2 shown in FIG. 76 is not regarded as a correctedcharacter, ‘10%’ is entered in the column ‘RELIABILITY’ on theintermediate process result table, and ‘BASIC’ is entered in the column‘PROCESS INSTRUCTION’ on the intermediate process result table.

‘DELETION LINE’ is entered in the column ‘COMPLETION OF PROCESS’ on theintermediate process result table, and ‘BASIC’ is entered in the column‘PROCESS ORDER’ on the intermediate process result table.

Next, since ‘BASIC’ is specified in the column ‘PROCESS ORDER’ of thecharacter box having the box number 3 on the intermediate process resulttable shown in FIG. 79, the process of the basic character recognizingunit 17 shown in FIG. 7 is performed on the pattern extracted from theone-character box having the box number 3 as shown in FIG. 76corresponding to ‘BASIC’ according to the process execution rule (C1).

The basic character recognizing unit 17 computes the distance betweenfeature vectors in a feature space by extracting the feature of an inputunknown character as shown in FIG. 35, representing the feature of theunknown character by a feature vector, and collating the vector with thefeature vector of each category preliminarily stored in the basicdictionary. Thus, the character category indicating the shortestdistance between the feature vectors is recognized as an unknowncharacter.

The basic character recognizing unit 17 computes the deformation of anunknown character by computing the number of convexity and concavity onthe outline of the unknown character. If the unknown character indicateslarge deformation and reduces the recognition ratio, then the knowledgetable 18 is referred to and a character recognizing process is performedby the detail identifying method shown in FIGS. 37 through 42.

The pattern extracted from the one-character box having the box number 3shown in FIG. 76 is recognized as the character category 3 with therecognition reliability of 95% in the character recognizing processperformed by the basic character recognizing unit 17, then ‘3’ isentered in the column ‘CHARACTER CODE’ on the intermediate processresult table, and ‘95%’ is entered in the column ‘RELIABILITY’ on theintermediate process result table.

‘BASIC’ is entered in the column ‘COMPLETION OF PROCESS’ on theintermediate process result table, and the column ‘PROCESS ORDER’ isblank on the intermediate process result table.

Next, since ‘BLACK-CHARACTER-BOX’ is first specified in the column‘PROCESS ORDER’ of the character box having the box number 4 on theintermediate process result table shown in FIG. 79, the process of thebox-touching character recognizing unit 13 shown in FIG. 7 is performedon the pattern extracted from the one-character box having the boxnumber 4 as shown in FIG. 76 corresponding to ‘BLACK-CHARACTER-BOX’according to the process execution rule (C1). Then, a characterrecognizing process is performed on a box-touching character.

If the recognition reliability of the pattern extracted from theone-character box having the box number 4 shown in FIG. 76 is computedas 15% in the character recognizing process performed by thebox-touching character recognizing unit 13, then the pattern extractedfrom the one-character box having the box number 4 as shown in FIG. 76is regarded as a non-character and ‘REJECT’ is entered in the column‘CHARACTER CODE’ on the intermediate process result table, and ‘15%’ isentered in the column ‘RELIABILITY’ on the intermediate process resulttable.

‘BLACK-CHARACTER-BOX’ is entered in the column ‘COMPLETION OF PROCESS’on the intermediate process result table, and the column ‘PROCESS ORDER’on the intermediate process result table is updated to ‘ONE-CHARACTERDELETION LINE’.

Next, since ‘ONE-CHARACTER DELETION LINE’ is specified in the column‘PROCESS ORDER’ of the character box having the box number 5-1 on theintermediate process result table shown in FIG. 79, the process of thedeletion line recognizing unit 26 shown in FIG. 3 is performed on thepattern extracted from the one-character box having the box number 5-1as shown in FIG. 76 corresponding to ‘ONE-CHARACTER DELETION LINE’according to the process execution rule (C1). Then, a recognizingprocess is performed on a pattern extracted as a candidate for acorrected character.

In the deletion line recognizing process performed by the deletion linerecognizing unit 26, the recognition reliability of the patternextracted from the one-character box having a box number 5-1 shown inFIG. 76 is computed as 95%. The pattern extracted from the box having abox number 5-1 shown in FIG. 76 is regarded as a corrected character,‘95%’ is entered in the column ‘RELIABILITY’ on the intermediate processresult table, and ‘DELETION LINE’ is entered in the column ‘COMPLETIONOF PROCESS’ on the intermediate process result table.

‘TERMINATION’ is entered in the column ‘PROCESS INSTRUCTION’ on theintermediate process result table, and the column ‘PROCESS ORDER’ isblank on the intermediate process result table.

Next, since ‘BLACK-CHARACTER-BOX’ is specified in the column ‘PROCESSORDER’ of the character box having the box number 5-2 on theintermediate process result table shown in FIG. 79, the process of thebox-touching character recognizing unit 13 shown in FIG. 7 is performedon the pattern extracted from the box having the box number 5-2 as shownin FIG. 76 corresponding to ‘BLACK-CHARACTER-BOX’ according to theprocess execution rule (C1). Then, a character recognizing process isperformed on a box-touching character.

The lower stroke of the pattern extracted from the character box havingthe box number 5-2 shown in FIG. 76 touches the character box. Since thepattern cannot be recognized with high reliability in the charactercompleting process shown in FIG. 43 or the character re-completingprocess shown in FIG. 44, a pair of a character and its misreadcharacter (2, 7) can be obtained by referring to the knowledge table 167shown in FIG. 49, and a character can be re-recognized by the areaemphasizing method shown in FIG. 51.

The pattern extracted from the box having the box number 5-2 shown inFIG. 76 is recognized as the character category 2 with the recognitionreliability of 95% in the character recognizing process performed by thebox-touching character recognizing unit 13, then ‘2’ is entered in thecolumn ‘CHARACTER CODE’ on the intermediate process result table, and‘95%’ is entered in the column ‘RELIABILITY’ on the intermediate processresult table.

‘BLACK-CHARACTER-BOX’ is entered in the column ‘COMPLETION OF PROCESS’on the intermediate process result table, and the column ‘PROCESS ORDER’is blank on the intermediate process result table.

Next, since ‘BASIC’ is specified in the column ‘PROCESS ORDER’ of thecharacter box having the box number 5-3 on the intermediate processresult table shown in FIG. 79, the process of the basic characterrecognizing unit 17 shown in FIG. 7 is performed on the patternextracted from the one-character box having the box number 5-3 as shownin FIG. 76 corresponding to ‘BASIC’ according to the process executionrule (C1). Then, a character recognizing process is performed on a basiccharacter.

The pattern extracted from the box having the box number 5-3 shown inFIG. 76 is recognized as the character category 6 with the recognitionreliability of 90% in the character recognizing process performed by thebasic character recognizing unit 17, then ‘6’ is entered in the column‘CHARACTER CODE’ on the intermediate process result table, and ‘90%’ isentered in the column ‘RELIABILITY’ on the intermediate process resulttable.

‘BASIC’ is entered in the column ‘COMPLETION OF PROCESS’ on theintermediate process result table, and the column ‘PROCESS ORDER’ isblank on the intermediate process result table.

Next, since ‘PLURAL-CHARACTER DELETION LINE’ is first specified in thecolumn ‘PROCESS ORDER’ of the character box having the box number 6-1-1on the intermediate process result table shown in FIG. 79, the processof the deletion line recognizing unit 26 shown in FIG, 7 is performedcorresponding to ‘PLURAL-CHARACTER DELETION LINE’ according to theprocess execution rule (C1). Then, a recognizing process is performed ondeletion lines.

In the deletion line recognizing process performed by the deletion linerecognizing unit 26, the recognition reliability of the deletion lineextracted from the table having a box number 6-1-1 shown in FIG. 76 iscomputed as 98%. The pattern extracted from the box having a box number6-1-1 shown in FIG. 76 is regarded as a corrected character,‘DELETIONLINE’ is entered in the column ‘CHARACTER CODE’, ‘98%’ is entered in thecolumn ‘RELIABILITY’ on the intermediate process result table, and‘DELETION LINE’ is entered in the column ‘COMPLETION OF PROCESS’ on theintermediate process result table.

According to the process execution rule (C 3), ‘TERMINATION’ is enteredin the column ‘PROCESS INSTRUCTION’ on the intermediate process resulttable, and the column ‘PROCESS ORDER’ is blank on the intermediateprocess result table.

Next, since ‘PLURAL-CHARACTER DELETION LINE’ is first specified in thecolumn ‘PROCESS ORDER’ of the character box having the box number 6-2-2on the intermediate process result table shown in FIG. 79, the processof the deletion line recognizing unit 26 shown in FIG. 7 is performedcorresponding to ‘PLURAL-CHARACTER DELETION LINE’ according to theprocess execution rule (C1). Then, a recognizing process is performed ondeletion lines.

In the deletion line recognizing process performed by the deletion linerecognizing unit 26, the recognition reliability of the deletion lineextracted from the table having the box number 6-2-2 shown in FIG. 76 iscomputed as 98%. The pattern extracted from the box having the boxnumber 6-2-2 shown in FIG. 76 is regarded as a corrected character,‘DELETION LINE’ is entered in the column ‘CHARACTER CODE’, ‘98%’ isentered in the column ‘RELIABILITY’ on the intermediate process resulttable, and ‘DELETION LINE’ is entered in the column ‘COMPLETION OFPROCESS’ on the intermediate process result table.

According to the process execution rule (C3), ‘TERMINATION’ is enteredin the column ‘PROCESS INSTRUCTION’ on the intermediate process resulttable, and the column ‘PROCESS ORDER’ is blank on the intermediateprocess result table.

The intermediate process result table shown in FIG. 80 is generated byperforming the above described processes. Since the process to be calledis entered in the column ‘PROCESS ORDER’ on the intermediate processresult table shown in FIG. 80, the processes are continued according tothe process execution rule (C1).

FIG. 81 shows the result obtained by continuing the recognizing processbased on the intermediate process result table shown in FIG. 80.

Since ‘ONE-CHARACTER DELETION LINE’ is specified for the first patternin the column ‘PROCESS ORDER’ of the character box having the box number1 on the intermediate process result table shown in FIG. 80, the processof the deletion line recognizing unit 26 shown in FIG. 1 is performed onthe first pattern extracted from the free-pitch character box having thebox number 1 as shown in FIG. 76 corresponding to ‘ONE-CHARACTERDELETION LINE’ according to the process execution rule (C1), and thecorrected character recognizing process is performed.

In the deletion line recognizing process performed by the deletion linerecognizing unit 26, the recognition reliability of the first patternextracted from the free-pitch character box having the box number 1shown in FIG. 76 is computed as 96%. The first pattern extracted fromthe free-pitch box having the box number 1 shown in FIG. 76 is regardedas a corrected character, ‘DELETION LINE’ is entered in the column‘CHARACTER CODE’ on the intermediate process result table, and ‘96%’ isentered in the column ‘RELIABILITY’ on the intermediate process resulttable, and ‘BLACK-CHARACTER-BOX/DELETION LINE’ is entered in the column‘COMPLETION OF PROCESS’ on the intermediate process result table.

Then, ‘TERMINATION’ is entered in the column ‘PROCESS INSTRUCTION’ onthe intermediate process result table, and the column ‘PROCESS ORDER’ onthe intermediate process result table is blank.

Next, ‘FREE-PITCH’ is specified in the column ‘PROCESS ORDER’ of thesecond pattern in the character box having the box number 1 on theintermediate process result table shown in FIG. 79. Therefore, theprocess of the character string recognizing unit 15 shown in FIG. 7 isperformed corresponding to the ‘FREE-PITCH’ on all patterns extractedfrom the free-pitch box having the box number 1 to recognize a characterwith the detection reliability of a character taken into accountaccording to the process execution rule (C4) when the columns ‘PROCESSORDER’ of all patterns in the character box having the box number 1contain ‘FREE-PITCH’ after extracting the second pattern from thefree-pitch box having the box number 1 shown in FIG. 76 and waiting forthe columns ‘PROCESS ORDER’ of all other patterns in the character boxhaving the box number 1 to contain ‘FREE-PITCH’.

Next, ‘FREE-PITCH’ is specified in the column ‘PROCESS ORDER’ of theeighth pattern in the character box having the box number 1 on theintermediate process result table shown in FIG. 79. Therefore, theprocess of the character string recognizing unit 15 shown in FIG. 7 isperformed corresponding to the ‘FREE-PITCH’ on all patterns extractedfrom the free-pitch box having the box number 1 to recognize a characterwith the detection reliability of a character taken into accountaccording to the process execution rule (C4) when the columns ‘PROCESSORDER’ of all patterns in the character box having the box number 1contain ‘FREE-PITCH’ after extracting the eighth pattern from thefree-pitch box having the box number 1 shown in FIG. 76 and waiting forthe columns ‘PROCESS ORDER’ of all other patterns in the character boxhaving the box number 1 to contain ‘FREE-PITCH’.

Then, the character recognizing process is performed by the characterstring recognizing unit 15 on all patterns extracted from the free-pitchbox having the box number 1 shown in FIG. 76 when the columns ‘PROCESSORDER’ of all patterns in the character box having the box number 1contain ‘FREE-PITCH’.

Relating to the first pattern extracted from the free-pitch box in thecharacter box having the box number 1 shown in FIG. 76, the recognizingprocess is performed by the character string recognizing unit 15 on thesecond through eighth patterns extracted from the free-pitch box havingthe box number 1 shown in FIG. 76 after removing the first patternextracted from the free-pitch box in the character box having the boxnumber 1 shown in FIG. 76 from the process objects of the characterstring recognizing unit 15 because ‘TERMINATION’ is entered in thecolumn ‘PROCESS INSTRUCTION’ of the first pattern in the character boxhaving the box number 1 on the intermediate process result table shownin FIG. 81.

The character string recognizing unit 15 computes the recognitionreliability based on the distance from the discriminant phase when acharacter is detected as shown in, for example, FIGS. 56 through 69, anddefines that the maximum product of character detection reliability andcharacter recognition reliability refers to a detected character.

In the recognizing process performed by the character string recognizingunit 15, the second pattern extracted from the free-pitch box having thebox number 1 shown in FIG. 76 is recognized as a character category ‘3’with recognition reliability of 95%. ‘3’ is entered in the column‘CHARACTER CODE’ on the intermediate process result table, and ‘95%’ isentered in the column ‘RELIABILITY’ on the intermediate process resulttable.

According to the process execution rule (C1),‘BLACK-CHARACTER-BOX/FREE-PITCH’ is entered in the column ‘COMPLETION OFPROCESS’ on the intermediate process result table, and the column‘PROCESS ORDER’ on the intermediate process result table becomes blank.According to the process execution rule (C4), ‘PERSONAL HANDWRITINGFEATURES’ is entered in the column ‘PROCESS INSTRUCTION’ on theintermediate process result table.

The eighth pattern extracted from the free-pitch box in the characterbox having the box number 1 shown in FIG. 76 is recognized as acharacter category ‘4’ with recognition reliability of 98%. ‘4’ isentered in the column ‘CHARACTER CODE’ on the intermediate processresult table, and ‘98%’ is entered in the column ‘RELIABILITY’ on theintermediate process result table.

According to the process execution rule (C1),‘BLACK-CHARACTER-BOX/FREE-PITCH’ is entered in the column ‘COMPLETION OFPROCESS’ on the intermediate process result table, and the column‘PROCESS ORDER’ on the intermediate process result table becomes blank.According to the process execution rule (C4), ‘PERSONAL HANDWRITINGFEATURES’ is entered in the column ‘PROCESS INSTRUCTION’ on theintermediate process result table.

The third pattern extracted from the free-pitch box in the character boxhaving the box number 1 shown in FIG. 76 is recognized as a charactercategory ‘2’. The fourth and fifth patterns extracted from thefree-pitch box in the character box having the box number 1 shown inFIG. 76 are integrated into a single character in the recognizingprocess by the character string recognizing unit 15 and recognized as acharacter category ‘7’. The sixth pattern extracted from the free-pitchbox in the character box having the box number 1 shown in FIG. 76 isrecognized as a character category ‘4’. The seventh pattern extractedfrom the free-pitch box in the character box having the box number 1shown in FIG. 76 is recognized as a character category ‘6’.

As a result, ‘7’ is input to the column ‘NUMBER OF CHARACTERS’ on theintermediate process result table shown in FIG. 81.

Next, since ‘BASIC’ is entered in the column ‘PROCESS ORDER’ of thecharacter box having the box number 2 on the intermediate process resulttable shown in FIG. 80, the process of the basic character recognizingunit 17 shown in FIG. 7 is performed corresponding to ‘BASIC’ on thepattern extracted from a one-character box having the box number 2 shownin FIG. 76 according to the process execution rule (C1).

In the character recognizing process performed by the basic characterrecognizing unit 17, the pattern extracted from the one-character box inthe character box having the box number 2 shown in FIG. 76 is recognizedas a character category ‘5’ with recognition reliability of 97%. ‘5’ isentered in the column ‘CHARACTER CODE’ on the intermediate processresult table, and ‘97%’ is entered in the column ‘RELIABILITY’ on theintermediate process result table. ‘DELETION LINE (YES 2)/BASIC’ isentered in the column ‘CALLING PROCESS’ on the intermediate processresult table. ‘DELETION LINE/BASIC’ is entered in the column ‘COMPLETIONOF PROCESS’ on the intermediate process result table. The column‘PROCESS ORDER’ on the intermediate process result table becomes blank.Therefore, according to the process execution rule (C4), ‘PERSONALHANDWRITING FEATURES’ is entered in the column ‘PROCESS INSTRUCTION’.

Since the column ‘PROCESS ORDER’ of the character box having the boxnumber 3 on the intermediate process result table shown in FIG. 80becomes blank, ‘PERSONAL HANDWRITING FEATURES’ is entered in the column‘PROCESS INSTRUCTION’ on the intermediate process result table accordingto the process execution rule (C4).

Next, since ‘ONE-CHARACTER DELETION LINE’ is entered in the column‘PROCESS ORDER’ of the character box having the box number 4 on theintermediate process result table shown in FIG. 80, the process of thedeletion line recognizing unit 26 shown in FIG. 76 is performedcorresponding to ‘ONE-CHARACTER DELETION LINE’ on the pattern extractedfrom the one-character box having the box number 4 shown in FIG. 76according to the process execution rule (C1). Thus, a recognizingprocess is performed on the pattern extracted as a candidate for acorrected character.

If the recognition reliability of the pattern extracted from theone-character box having the box number 4 shown in FIG. 76 is computedas 95% in the deletion line recognizing process performed by thedeletion line recognizing unit 26, then the pattern extracted from theone-character box having the box number 4 as shown in FIG. 76 isregarded as a corrected character and ‘95%’ is entered in the column‘RELIABILITY’ on the intermediate process result table, and‘BLACK-CHARACTER-BOX/DELETION LINE’ is entered in the column ‘COMPLETIONOF PROCESS’ on the intermediate process result table.

‘TERMINATION’ is entered in the column ‘PROCESS INSTRUCTION’ on theintermediate process result table, and the column ‘PROCESS ORDER’ isblank on the intermediate process result table.

Since ‘TERMINATION’ is entered in the column ‘PROCESS INSTRUCTION’ ofthe box number having the box number 5-1 on the intermediate processresult table shown in FIG. 80, no process is performed on the patternextracted from the character box having the box number 5-1 shown in FIG.76.

Since the column ‘PROCESS ORDER’ of the character box having the boxnumber 5-2 on the intermediate process result table shown in FIG. 80becomes blank, ‘PERSONAL HANDWRITING FEATURES’ is entered in the column‘PROCESS INSTRUCTION’ on the intermediate process result table accordingto the process execution rule (C4).

Since the column ‘PROCESS ORDER’ of the character box having the boxnumber 5-3 on the intermediate process result table shown in FIG. 80becomes blank, ‘PERSONAL HANDWRITING FEATURES’ is entered in the column‘PROCESS INSTRUCTION’ on the intermediate process result table accordingto the process execution rule (C4).

Since ‘TERMINATION’ is entered in the column ‘PROCESS INSTRUCTION’ ofthe box number having the box number 6-1-1 on the intermediate processresult table shown in FIG. 80, no process is performed on the patternextracted from the character box having the box number 6-2-2 shown inFIG. 76.

Since ‘TERMINATION’ is entered in the column ‘PROCESS INSTRUCTION’ ofthe box number having the box number 6-2-2 on the intermediate processresult table shown in FIG. 80, no process is performed on the patternextracted from the character box having the box number 6-1-1 shown inFIG. 76.

The intermediate process result table shown in FIG. 81 is generated byperforming the above described processes. Since the ‘PERSONALHANDWRITING FEATURES’ is entered in a column ‘PROCESS INSTRUCTION’ onthe intermediate process result table shown in FIG. 81, the processesare continued according to the process execution rule (C5).

FIG. 82 shows the result obtained by continuing the recognizing processbased on the intermediate process result table shown in FIG. 81.

Since ‘TERMINATION’ is entered in the column ‘PROCESS INSTRUCTION’ ofthe first pattern in the character box having the box number 1 on theintermediate process result table shown in FIG. 80, no process isperformed on the first pattern extracted from the free-pitch box havingthe box number 1 shown in FIG. 76.

Since ‘PERSONAL HANDWRITING FEATURES’ is entered in the column ‘PROCESSINSTRUCTION’ of the second pattern in the character box having the boxnumber 1 on the intermediate process result table shown in FIG. 81, theprocess of the unique character analyzing unit 23 shown in FIG. 7 isperformed corresponding to ‘PERSONAL HANDWRITING FEATURES’ on the secondpattern extracted from the free-pitch box having the box number 1 shownin FIG. 76 according to the process execution rule (C5).

For example, the unique character analyzing unit 23 clusters thecharacters handwritten by the same writer into categories as shown inFIGS. 72 through 75. The second cluster, which is close to the firstcluster of written characters obtained by the clustering process,belongs to a category different from that of the first cluster, and hasa smaller number of elements, is integrated into the first cluster sothat the category of the handwritten characters belonging to the secondcluster can be amended to the category of the first clusters.

In the analyzing process performed by the unique character analyzingunit 23, the second pattern extracted from the free-pitch box having thecharacter box number 1 shown in FIG. 76 is recognized as charactercategory 3 with the recognition reliability of 97%. ‘3’ is entered inthe column ‘CHARACTER CODE’ on the intermediate process result table,and ‘97%’ is entered in the column ‘RELIABILITY’ on the intermediateprocess result table.

‘BLACK-CHARACTER-BOX/FREE-PITCH/PERSONAL HANDWRITING FEATURES’ isentered in the column ‘COMPLETION OF PROCESS’, and ‘TERMINATION’ isentered in the column ‘PROCESS INSTRUCTION’.

Since ‘PERSONAL HANDWRITING FEATURES’ is entered in the column ‘PROCESSINSTRUCTION’ of the eighth pattern in the character box having the boxnumber 1 on the intermediate process result table shown in FIG. 81, theprocess of the unique character analyzing unit 23 shown in FIG. 7 isperformed corresponding to ‘PERSONAL HANDWRITING FEATURES’ on the eighthpattern extracted from the free-pitch box having the box number 1 shownin FIG. 76 according to the process execution rule (C5).

In the analyzing process performed by the unique character analyzingunit 23, the eighth pattern extracted from the free-pitch box having thecharacter box number 1 shown in FIG. 76 is recognized as charactercategory 4 with the recognition reliability of 98%. ‘4’ is entered inthe column ‘CHARACTER CODE’ on the intermediate process result table,and ‘98%’ is entered in the column ‘RELIABILITY’ on the intermediateprocess result table.

‘BLACK-CHARACTER-BOX/FREE-PITCH/PERSONAL HANDWRITING FEATURES’ isentered in the column ‘COMPLETION OF PROCESS’, and ‘TERMINATION’ isentered in the column ‘PROCESS INSTRUCTION’.

Since ‘PERSONAL HANDWRITING FEATURES’ is entered in the column ‘PROCESSINSTRUCTION’ in the character box having the box number 2 on theintermediate process result table shown in FIG. 81, the process of theunique character analyzing unit 23 shown in FIG. 7 is performedcorresponding to ‘PERSONAL HANDWRITING FEATURES’ on the patternextracted from the one-character box having the box number 2 shown inFIG. 76 according to the process execution rule (C5).

In the analyzing process performed by the unique character analyzingunit 23, the pattern extracted from the one-character box having thecharacter box number 2 shown in FIG. 76 is recognized as charactercategory 5 with the recognition reliability of 97%. ‘5’ is entered inthe column ‘CHARACTER CODE’ on the intermediate process result table,and ‘97%’ is entered in the column ‘RELIABILITY’ on the intermediateprocess result table.

‘DELETION LINE/BASIC/PERSONAL HANDWRITING FEATURES’ is entered in thecolumn ‘COMPLETION OF PROCESS’, and ‘TERMINATION’ is entered in thecolumn ‘PROCESS INSTRUCTION’.

Since ‘PERSONAL HANDWRITING FEATURES’ is entered in the column ‘PROCESSINSTRUCTION’ in the character box having the box number 3 on theintermediate process result table shown in FIG. 81, the process of theunique character analyzing unit 23 shown in FIG. 7 is performedcorresponding to ‘PERSONAL HANDWRITING FEATURES’ on the patternextracted from the one-character box having the box number 3 shown inFIG. 76 according to the process execution rule (C5).

In the analyzing process performed by the unique character analyzingunit 23, the pattern extracted from the one-character box having the boxnumber 3 shown in FIG. 76 is recognized as character category 3 with therecognition reliability of 97%. ‘3’ is entered in the column ‘CHARACTERCODE’ on the intermediate process result table, and ‘97%’ is entered inthe column ‘RELIABILITY’ on the intermediate process result table.

‘BASIC/PERSONAL HANDWRITING FEATURES’ is entered in the column‘COMPLETION OF PROCESS’, and ‘TERMINATION’ is entered in the column‘PROCESS INSTRUCTION’.

Since ‘TERMINATION’ is entered in the column ‘PROCESS INSTRUCTION’ ofthe box number having the box number 4 on the intermediate processresult table shown in FIG. 81, no process is performed on the patternextracted from the character box having the box number 4 shown in FIG.76.

Since ‘TERMINATION’ is entered in the column ‘PROCESS INSTRUCTION’ ofthe box number having the box number 5-1 on the intermediate processresult table shown in FIG. 81, no process is performed on the patternextracted from the character box having the box number 5-1 shown in FIG.76.

Since ‘PERSONAL HANDWRITING FEATURES’ is entered in the column ‘PROCESSINSTRUCTION’ in the character box having the box number 5-2 on theintermediate process result table shown in FIG. 81, the process of theunique character analyzing unit 23 shown in FIG. 7 is performedcorresponding to ‘PERSONAL HANDWRITING →FEATURES’ on the patternextracted from the character box having the box number 5-2 shown in FIG.76 according to the process execution rule (C5).

In the analyzing process performed by the unique character analyzingunit 23, the pattern extracted from the character box having the boxnumber 5-2 shown in FIG. 76 is recognized as character category 2 withthe recognition reliability of 97%. ‘2’ is entered in the column‘CHARACTER CODE’ on the intermediate process result table, and ‘97%’ isentered in the column ‘RELIABILITY’ on the intermediate process resulttable.

‘BLACK-CHARACTER-BOX/PERSONAL HANDWRITING FEATURES’ is entered in thecolumn ‘COMPLETION OF PROCESS’, and ‘TERMINATION’ is entered in thecolumn ‘PROCESS INSTRUCTION’.

Since ‘PERSONAL HANDWRITING FEATURES’ is entered in the column ‘PROCESSINSTRUCTION’ in the character box having the box number 5-3 on theintermediate process result table shown in FIG. 81, the process of theunique character analyzing unit 23 shown in FIG. 7 is performedcorresponding to ‘PERSONAL HANDWRITING FEATURES’ on the patternextracted from the character box having the box number 5-3 shown in FIG.76 according to the process execution rule (C5).

In the analyzing process performed by the unique character analyzingunit 23, the pattern extracted from the character box having the boxnumber 5-3 shown in FIG. 76 is recognized as character category 4 withthe recognition reliability of 96%. ‘4’ is entered in the column‘CHARACTER CODE’ on the intermediate process result table, and ‘96%’ isentered in the column ‘RELIABILITY’ on the intermediate process resulttable.

‘BASIC/PERSONAL HANDWRITING FEATURES’ is entered in the column‘COMPLETION OF PROCESS’, and ‘TERMINATION’ is entered in the column‘PROCESS INSTRUCTION’.

Since ‘TERMINATION’ is entered in the column ‘PROCESS INSTRUCTION’ ofthe box number having the box number 6-1-1 on the intermediate processresult table shown in FIG. 81, no process is performed on the patternextracted from the character box having the box number 6-1-1 shown inFIG. 76.

Since ‘TERMINATION’ is entered in the column ‘PROCESS INSTRUCTION’ ofthe box number having the box number 6-2-2 on the intermediate processresult table shown in FIG. 81, no process is performed on the patternextracted from the character box having the box number 6-2-2 shown inFIG. 76.

The intermediate process result table shown in FIG. 82 is generated byperforming the above described processes. Since ‘TERMINATION’ is enteredin the column ‘PROCESS INSTRUCTION’ on the intermediate process resulttable shown in FIG. 82 for all process objects, all processes terminateaccording to the process execution rule (C6).

As described above, the character recognizing unit 12 and non-characterrecognizing unit 25 perform appropriate recognizing processes to processthe state of the input image recognized by the environment recognizingsystem 11 according to the embodiments of the present invention.

For example, when the environment recognizing system 11 extracts acharacter. touching the ruled line, it uses the box-touching characterrecognizing unit 13 for exclusively performing a recognizing process ona box-touching character. When the environment recognizing system 11extracts a free-pitch character string, it uses the character stringrecognizing unit 15 for exclusively performing a recognizing process ona free-pitch character string. When the environment recognizing system11 extracts an obscure character, it uses the obscure characterrecognizing unit 19 for exclusively performing a recognizing process onan obscure character. When the environment recognizing system 11extracts a deformed character, it uses the deformed characterrecognizing unit 21 for exclusively performing a recognizing process ona deformed character. When the environment recognizing system 11extracts a non-character, it uses the non-character recognizing unit 25for exclusively performing a recognizing process on a non-character.

The reliability on the recognition result from the character recognizingunit 12 or non-character recognizing unit 25 is computed. For acharacter or non-character with low reliability, the environmentrecognizing system 11, character recognizing unit 12, and non-characterrecognizing unit 25 mutually feed back data to re-perform otherprocesses. When high reliability is obtained or there are no executableprocesses to be performed, the entire process terminates.

Thus, according to the embodiments of the present invention, arecognizing process can be performed with the features and identifyingmethods to be used when characters are recognized adaptively amendeddepending on the environment in which characters are written. Therefore,a high precision character recognition can be realized corresponding tovarious environments of documents and forms.

Furthermore, character recognition results can be confirmed with highreliability by outputting only character codes as recognition results,simultaneously outputting environment recognition results and characterrecognition results, and outputting character recognition results whenthe environment recognition results and character recognition resultsmatch each other.

Since the non-character recognizing unit 25 can be provided exclusivelyfor performing a non-character recognizing process independent of acharacter recognizing process, the reliability in the character andnon-character recognizing processes can be improved.

Furthermore, since a recognizing process can be performed independentlyof the environment, the recognition reliability can be improved byincreasing the volume of the dictionary and knowledge in eachrecognizing process.

Described below is the pattern recognizing apparatus according to thethird embodiment of the present invention.

The pattern recognizing apparatus according to the third embodimentcorrectly rejects deletion lines apparently indicating the deletion of acharacter to prevent it from being mis-read, but does not mistakenlyreject a character other than a character with deletion lines, therebyrecognizing a character with high reliability without a heavy load to auser.

The third embodiment shown in FIG. 83 comprises an image input unit 411for receiving an image, detecting a character pattern from the inputimage, and pre-processing the detected pattern; a deletion linecharacter discriminating unit 412 for discriminating a character withdeletion lines of either simple deletion lines formed simply byhorizontal lines indicating the deletion of the character or complicateddeletion lines formed by complicated strokes, or a normal characterwithout deletion lines; and an identifying unit 413 for identifying acharacter.

‘Inputting an image’ refers to inputting an original image representinga character. For example, when a character on a form is to berecognized, the character pattern on the form is transmitted to the readarea of an opto-electrical converter to read the character pattern,convert it into an electrical signal, and output a digital signalthrough binarization, etc.

‘Detecting a character’ refers to detecting only a character portionfrom the form data of the input digital image and separating eachcharacter from others.

‘Pre-processing’ refers to removing noises, standardizing, the position,size, pose, line-thinness, etc. of a character.

Discriminating a character with deletion lines can be a pre-process.

A ‘character with deletion lines’ refers to a character provided withdeletion lines indicating the deletion of the character. According tothe third embodiment, deletion lines can be simple deletion lines orcomplicated deletion lines.

A ‘simple deletion line’ refers to a deletion line formed by ahorizontal line assumed to indicate the deletion of a character. Thesimple deletion line can be one or more horizontal lines.

A‘horizontal line’ is not always a correctly-horizontal line, butincludes a line drawn in the horizontal direction with permittedallowance in slope.

Thus, a character with an apparent horizontal deletion line can bedeleted without fail and mis-reading can be reduced.

In the embodiment described later, horizontal deletion lines arediscriminated by a contiguous projection.

A‘complicated deletion line’ refers to a complicated line or strokedrawn on a character to be deleted. To determine whether or not a lineis a complicated deletion line, the amount of complexity is extracted bythe pattern recognizing apparatus shown in FIG. 84.

‘Identifying a character’ refers to extracting a feature, inputting theobtained feature vector, and computing the state of matching between thefeature vector and a standard feature vector, that is, a preliminarilystored identifying dictionary. The category of the standard featurevector indicating the highest similarity is output as a recognitionresult.

According to the present embodiment, a discriminating process isperformed on a character with a simple deletion line or a complicateddeletion line so that a normal character can be identified.

Therefore, such deletion lines as a predetermined horizontal line, anapparent deletion line simple and a little irregular from thepredetermined horizontal line, and an apparent deletion line complicatedand quite different from the simple deletion line can be discriminatedas deletion lines without fail.

FIG. 84 is a block diagram showing the configuration of the patternrecognizing apparatus according to the fourth embodiment of the presentinvention.

In FIG. 84, the deletion line character discriminating unit 412comprises a complicated deletion line character discrimination processunit 414 for discriminating a character with a complicated deletion lineand a simple deletion line character discriminating unit 415 fordiscriminating a character with a simple deletion line by determiningwhether or not a candidate for a character with a simple deletion linecan be identified as a character when the candidate for the simpledeletion line is removed from the candidate for the character with thesimple deletion line when the plural-character deletion line characterdetermination process unit 414 determines that the pattern is not acharacter with a complicated deletion line.

According to the pattern recognizing apparatus shown in FIG. 84, acharacter with a simple deletion line can be discriminated only when itis determined that the character is not provided with a complicateddeletion line after first determining whether or not the character is acharacter with a complicated deletion line. Therefore, the patternrecognizing apparatus avoids misreading a character for a characterwithout a deletion line when it cannot be identified as a characterafter removing a simple deletion line from it, thereby discriminatingwith high reliability a candidate for a character with a deletion line.

FIG. 85 is a block diagram showing the configuration of the patternrecognizing apparatus according to the fifth embodiment of the presentinvention.

In FIG. 85, the deletion line character discriminating unit 412comprises a character form information extracting unit 416 forextracting character form information comprising at least one of thepicture element number histogram, obtained by the contiguous projectionin which the black picture elements in the object horizontal scanningline and the contiguous lines in a predetermined range above and belowthe object line are added up and counted in the horizontal direction,and the amount of complexity indicating the complexity of drawings fromthe detected and pre-processed character pattern; a deletion linecharacter candidate discriminating unit 417 for discriminating acandidate. for a character with a simple deletion line or a characterwith a complicated deletion line according to the character forminformation extracted by the character form information extracting unit416; and a deletion line character determining unit 418 for determininga character with a simple deletion line when the character can beidentified as a character by the identifying unit 413 after removing acandidate for a deletion line from the candidate for a character with asimple deletion line.

The ‘character form information’ comprises at least one of the‘picture-element-number histogram obtained by a contiguous projectionperformed in the horizontal direction’ and ‘amount of complexity’

A ‘predetermined range’ refers to a range of contiguous lines above andbelow the object horizontal line. For example, the range can contain thethree lines, that is, the object horizontal line and the linesimmediately above and below the object line according to the embodimentdescribed later.

If the ‘predetermined range’ is set narrow, then only lines in thehorizontal direction can be recognized as simple deletion lines. If itis set larger, then even a line making an angle with the horizontal linecan be recognized as a simple deletion line. However, if it is set toolarge, the peak of the picture element histogram becomes dull and thehorizontal lines cannot be correctly determined. Therefore, the range isappropriately set depending on experiences and experiments.

In a ‘contiguous projection in the horizontal direction’, the number ofblack picture elements is counted by adding up the black pictureelements in the lines in a predetermined range containing an objecthorizontal scanning line and the contiguous lines above and below theobject line.

An ‘amount of complexity’ can be represented by, for example, the linedensity in a predetermined direction, Euler number, number of blackpicture elements, etc.

According to the present embodiment, a character with a simple deletionline is discriminated by the picture-element-number histogram obtainedby a contiguous projection in the horizontal direction, and a characterwith a complicated deletion line is discriminated by the amount ofcomplexity.

The above described amount of complexity or the picture-element-numberhistogram obtained by the contiguous projection in the horizontaldirection can be obtained objectively, easily, rapidly, and with highreliability. Therefore, a character with a deletion line can be easilydiscriminated at a high speed and with high reliability.

FIG. 86 is a block diagram showing the configuration of the patternrecognizing apparatus according to the sixth embodiment of the presentinvention.

In FIG. 86, the complicated deletion line character discriminationprocess unit 414 comprises a complexity amount extracting unit 419 forextracting the amount of complexity indicating the complexity ofdrawings, and a complicated deletion line character discriminating unit420 for determining a character with a complicated deletion line basedon the extracted amount of complexity. The simple deletion linecharacter discriminating unit 415 comprises a picture element numberhistogram computing unit 421 for computing the picture element numberhistogram by the contiguous projection in which the black pictureelements in the object horizontal scanning line and the contiguous linesin a predetermined range above and below the object line are added upand counted in the horizontal direction; a simple deletion linecharacter candidate discriminating unit 422 for discriminating acandidate for a character with a simple deletion line based on thecomputed number of picture elements, and a simple deletion linecharacter determining unit 423 for determining a character with a simpledeletion line when the character can be identified as a character by theidentifying unit 413 after removing a candidate for a deletion line fromthe candidate for a character with a simple deletion line.

According to the pattern recognizing apparatus shown in FIG. 86, acharacter with a deletion line can be discriminated based on the pictureelement number histogram obtained by the contiguous projection in thehorizontal direction and the amount of complexity.

Therefore, in addition to the effect obtained by the pattern recognizingapparatus shown in FIG. 84, the character with a deletion line can bediscriminated with high reliability.

FIG. 87 is a block diagram showing the configuration of the patternrecognizing apparatus according to the seventh embodiment of thepresent, invention.

In FIG. 87, the simple deletion line character determining unit 423comprises a deletion line removing unit 424 for removing a candidate fora deletion line from a discriminated candidate for a character with asimple deletion line and transmitting the result to the identifying unit413; a storage unit 426 for storing an original image of a candidate fora character with a simple deletion line before removing the candidatefor a deletion line; and a deletion line character determining unit 425for defining a candidate for a character with a simple deletion line asa character with a simple deletion line when the candidate for thecharacter with a simple deletion line can be identified as a charactereven after removing the deletion line from the character, and fordefining the candidate for a character with a simple deletion linestored in the storage unit 426 as a normal character and transmitting itto the identifying unit 413 when the candidate for the character with asimple deletion line cannot be identified as a character after removingthe deletion line from the character.

According to the pattern recognizing apparatus shown in FIG. 87, theoriginal image before removing by the deletion line removing unit 424 acandidate for a deletion line from a candidate for a character with adeletion line is temporarily stored in the storage unit 426.

Therefore, if the candidate, from which the candidate for a deletionline is removed, for a character with a deletion line cannot beidentified as a character, then the original image of the candidate fora character with a deletion line is read from the storage unit 426 andtransmitted to the identifying unit 413 for identification of acharacter. Thus, a character can be identified at a high speed with asimple configuration according to the present invention.

FIG. 88 is a flowchart showing the operations of the pattern recognizingapparatus according to the eighth embodiment of the present invention.

In FIG. 88, an image is input, a character pattern is detected from theinput image, and the result is pre-processed in step S301. Then, adiscriminating process is performed in step S302 on the detected andpre-processed character pattern as to whether the current character is acharacter with a deletion line provided with either a simple deletionline formed by only a horizontal line or a complicated deletion linedrawn by a complicated stroke to indicate the deletion of the characteror a normal character without a deletion line. Thus, a normal characterwithout a deletion line can be identified as a character in step S303.

According to the pattern recognizing apparatus shown in FIG. 88, it isdetermined whether the current character is a character with a deletionline provided with either a simple deletion line or a complicated lineor a normal character without a deletion line. Thus, a normal charactercan be identified as a character.

Therefore, since the user can recognize a character with a simpleconfiguration and high reliability because a character with an apparentdeletion line can be removed without fail even if the user is notfamiliar with the deletion lines or is not provided with any instructionabout deletion lines.

Thus, the restrictions on the entries of a form, etc. can be reduced andthe load to the user can also be reduced considerably.

FIG. 89 is a flowchart showing the operations of the pattern recognizingapparatus according to the ninth embodiment of the present invention.

In FIG. 89, an image is input, a character pattern is detected from theinput image, and the result is pre-processed in step S304. Then, adiscriminating process is performed in step S305 to discriminate acharacter with a complicated deletion line having a complicated form onthe detected and pre-processed character pattern. If it is determinedthat the current character is not a character with a complicateddeletion line, then it is determined whether the current character is acharacter with a simple deletion line formed by only a horizontal lineto indicate the deletion of the character or a normal character withouta deletion line in step S306. Thus, a normal character without adeletion line can be identified as a character in step S307.

Thus, according to the pattern recognizing apparatus shown in FIG. 89,when a discriminating process is performed on a character with adeletion line, it is first performed on a character with a complicateddeletion line. Only if it is determined that the current character isnot a character with a complicated deletion line, the discriminatingprocess is performed on a candidate for a character with a simpledeletion line.

Therefore, a character can be discriminated efficiently and rapidly

A candidate for a character with a simple deletion line is defined as acharacter with a simple deletion line when the character can beidentified as a character even after removing a candidate for a deletionline from the candidate for a character. If the candidate for thecharacter cannot be identified as a character, it is defined as a normalcharacter.

Since a complicated deletion line has been removed, a character with acomplicated deletion line is not mixed among normal characters and isnot misread for a wrong character.

Furthermore, since it is determined whether or not a candidate for acharacter with a simple deletion line can be identified as a character,the determination as to whether or not the candidate is a character witha deletion line can be made correctly with high reliability.

FIG. 90 is a flowchart showing the operations of the pattern recognizingapparatus according to the tenth embodiment of the present invention.

In FIG. 90, an image is input, a character pattern is detected from theinput image, and the detected result is pre-processed in step S311. Theamount of complexity of the detected and pre-processed character iscomputed in step S312. Based on the amount of complexity, adiscriminating process about a character with a complicated deletionline represented by a complicated stroke is performed in step S313. Ifit is determined that the character is not provided with a complicateddeletion line, then the picture element number histogram is obtained bythe contiguous projection in which the black picture elements in theobject horizontal scanning line and the contiguous lines in apredetermined range above and below the object line are added up andcounted in the horizontal direction in step S314. Then, thediscriminating process is performed about the candidate for a characterwith a simple deletion line in step S315. If a candidate for a deletionline is removed from a candidate for a character with a simple deletionline and the result can be identified as a character, then the candidateis defined as a character with a simple deletion line. If it cannot beidentified as a character, the candidate is defined as a normalcharacter in step S316, and it is determined in step S317 whether or notthe defined normal character can be identified as a character.

Thus, according to the pattern recognizing apparatus shown in FIG. 90, adiscriminating process is performed about a character with a simpledeletion line or a character with a complicated deletion line byextracting the amount of complexity or performing the contiguousprojection.

Therefore, a discriminating process can be easily and correctlyperformed with a simple configuration at a high speed.

FIG. 91 is a flowchart showing the operations of the pattern recognizingapparatus according to the eleventh embodiment of the present invention.

In FIG. 91, the amount of complexity such as an Euler number, the linedensity, the density of black picture elements, etc. is computed in stepS321 about the detected and pre-processed character. A determination asto whether or not the current character is provided with a complicateddeletion line is made using a predetermined threshold in step S322 onthe computed amount of complexity. If it is determined that the currentcharacter is a character with a complicated deletion line, then a rejectcode is output in step S323. If it is not determined that the currentcharacter is provided with a complicated deletion line, then ahorizontal direction continguous projection histogram is computed instep S314, and it is determined whether or not a candidate for acharacter with a simple deletion line exists in step S351. If there is acandidate for a deletion line, it is removed as a candidate for acharacter with a simple deletion line in step S352. If the candidate fora deletion line is removed from the candidate for a character with asimple deletion line, and the result can be identified as a character,the candidate for a character with a simple deletion line is defined asa character with a simple deletion line. If the result cannot beidentified as a character, it is determined whether or not the currentcharacter is a character with a simple deletion line in step S351 whenit is defined as determines a normal character in step S316. If it is acandidate for a character with a deletion line, then the candidate for adeletion line is removed in step S352, and the candidate for a characterwith deletion line from which the candidate for the deletion line isremoved is checked whether or not it can be identified as a character.If it can be identified as a character, it is defined as a characterwith a simple deletion line and a reject code is output (step S355). Ifit cannot be identified as a character, it is defined as a normalcharacter (steps S353 and S354). In the process (S317) of identifying anormal character, the defined normal character is checked whether or notit can be identified as a character in steps S361 and S262. If it cannotbe identified as a character, then a reject code is output in step S363.If it can be identified as a character, the result of the recognition isoutput (step S364).

FIG. 92 is a block diagram showing the configuration of the patternrecognizing apparatus according to the twelfth embodiment of the presentinvention.

In FIG. 92, the apparatus comprises an optical reading unit 430 foroptically reading transmitted character pattern entered in a form,converting the character pattern into an electrical signal, andoutputting a digital signal through binarization, etc.; a transmittingunit 431 for transmitting the form to the optical reading area of theoptical reading unit 430; a dictionary 432 storing the standard featurevector of a character; a display unit for displaying a character on ascreen; an output unit 433 for printing a character on paper; anoperating unit 434 for performing various operations; and a CPU andmemory 435 having various functions for character recognition.

As shown in FIG. 92, the CPU and memory 435 comprises a characterdetecting and preprocessing unit 436 for detecting and pre-processing acharacter from an input image; a complicated deletion line characterdiscrimination process unit 437 for performing a discriminating processon a character with a complicated deletion line represented by acomplicated stroke; a simple deletion line character discriminating unit438 for performing a discriminating process on a candidate for acharacter with a simple deletion line formed only by a horizontal lineindicating the deletion of a character when it is determined by thecomplicated deletion line character determination process unit 437 thatthe character is not provided with a complicated deletion line; anidentifying unit 439 for identifying a character; and a result outputdirecting unit 440 for rejecting the character if it is discriminated asa character with a deletion line, outputting a character identificationresult if the character is discriminated as a normal character, andgiving a direction to output the rejection if the character isdiscriminated as a normal character but cannot be identified as acharacter.

As shown in FIG. 92, the character detecting and preprocessing unit 436comprises a character detecting unit 441 for detecting only thecharacter portion from the form image of an input digital image andseparating characters from each other, and a pre-processing unit 442 forremoving a noise from a detected character signal and standardizing theposition, size, etc. of a character.

The optical reading unit 430, transmitting unit 431, and characterdetecting and preprocessing unit 436 correspond to an image input unit.

The complicated deletion line character discrimination process unit 437comprises a complexity amount extracting unit 443 for extracting theamount of complexity from a character pattern; and a complicateddeletion line character discriminating unit 444 for discriminatingwhether or not the character pattern is a character with a complicateddeletion line based on the extracted amount of complexity.

The ‘amount of complexity’ can be the line density in a predetermineddirection, an Euler number, black picture element density, etc.

The ‘line density in a predetermined direction’ refers to a valueobtained by counting the portions changing from white picture elementsinto black picture elements (or black picture elements into whitepicture elements) when an image in a rectangle is scanned in apredetermined direction. For example, if a character pattern is formedby a character ‘2’ with a deletion line 501 as indicated by (A) in FIG.93, the line density in the vertical direction as indicated by (D) inFIG. 93. is 6.

The ‘predetermined direction’ normally refers to the vertical orhorizontal direction to a character.

The ‘Euler number’ E is represented by E=C−H by subtracting the number Hof holes from the number of link elements C where C indicates the numberof link elements connected to one another in an image, and H indicatesthe number of holes in the image.

For example, if a character pattern is formed by a character ‘2’ with adeletion line 502 as indicated by (B) in FIG. 93, the Euler number E=−4as indicated by (E) in FIG. 93.

The ‘black picture element density’ D=B/S indicates the ratio of thearea S (total number of picture elements) of a circumscribing rectangleof an object image to the total number B of black picture element in thecircumscribing rectangle of the object image.

For example, if a character pattern is formed by a character ‘2’ with adeletion line 503 as indicated by (C) in FIG. 93, the black pictureelement density D is represented by D=B/S as indicated by (F) in FIG.93.

(A) through (C) in FIG. 93 shows examples of characters with complicateddeletion line.

A complicated deletion line character discriminating unit 444 performs adiscriminating process based on the amount of complexity such as theline density, Euler number, or black picture element density, etc. ofthe extracted feature or an appropriate combination of them.

General tendency of the extracted amount of complexity and a normalcharacter or a character with a deletion line is as follows.

feature/tend- normal character character with ency deletion line maximumline small large density Euler small abstract negative value numbervalue (2 ˜ -1) large absolute value back picture small large elementdensity

It is determined based on the amount of complexity whether or not thecurrent character is provided with a deletion line by preliminarilysetting a threshold, etc.

The simple deletion line character discriminating unit 438 shown in FIG.92 comprises a picture element histogram computing unit 445 forcomputing the picture element number histogram by the contiguousprojection in which the black picture elements in the object horizontalscanning line and the contiguous lines in a predetermined range aboveand below the object line are added up and counted in the horizontaldirection; a simple deletion line character candidate discriminatingunit 446 for performing a discriminating process about a candidate for acharacter with a simple deletion line based on the computed number ofpicture elements; and a simple deletion line character determining unit447 for determining a character with a simple deletion line if theidentifying unit 439 identifies a character when a candidate for adeletion line is removed from the discriminated candidate for acharacter with a simple deletion line.

The ‘contiguous projection in the horizontal direction’ refers to addingup and counting in the horizontal direction the black picture elementsin the object horizontal scanning line and the contiguous lines in apredetermined range above and below the object line.

According to the aspect of the present embodiment, a predetermined rangecontains 3 lines.

The simple deletion line character candidate discriminating unit 446determines a candidate for a character with a simple horizontal deletionline having a peak whose picture element number histogram exceeds apredetermined threshold, and recognizes the corresponding line as acandidate for a deletion line.

As shown in FIG. 71, if the candidate for a character with a deletionline determined by the simple deletion line character candidatediscriminating unit 446 and stripped of the candidate for a deletionline by the deletion line removing unit 450 can be identified as acharacter, then it is defined as a character with a deletion line by thesimple deletion line character determining unit 448. If it cannot beidentified as a character, then it is defined as a normal character(character without a deletion line). A candidate for a deletion line isremoved by the deletion line removing unit 450 by an existing imageprocessing method.

As shown in FIG. 92, the simple deletion line character determining unit447 comprises a deletion line removing unit 450 for removing a candidatefor a deletion line from the discriminated candidate for a characterwith a simple deletion line and transmitting the result to theidentifying unit 439; the simple deletion line character determiningunit 448 for determining the candidate for a character with a simpledeletion line as a character with a simple deletion line when thediscriminated candidate for a character with a simple deletion line isrecognized as a character, and determining it as a normal character andtransmitting it to the identifying unit 439 when it is not recognized asa character; and a character candidate storage unit 449 for storing thecandidate for a character with a deletion line removed by the deletionline removing unit 450.

Furthermore, the identifying unit 439 comprises a feature extractingunit 451 for extracting the feature value of each character pattern andcompressing data; and a dictionary collating unit 452 for collating acharacter pattern with a dictionary, that is, a standard feature vectorof each character type.

Then, the operations of the pattern recognizing apparatus shown in FIG.92 are described by referring to FIGS. 94 and 95.

As shown in FIG. 94, the transmitting unit 431 transmits a formcontaining entered characters to the reading area of the optical readingunit 430 and has an OCR input the form in step SJ1.

In step SJ2, the optical reading unit 430 converts a character patternon the form into an electrical signal in the opto-electrical conversion,and outputs it as a digital signal through binarization, etc.

The form can contain entered characters and their character boxes in thesame color.

In step SJ3, the character detecting unit 441 detects a characterportion from the digital signal and separates characters from oneanother. The pre-processing unit 442 standardizes the position, size,slope line thinness, etc.

In step SJ4, a complicated deletion line character discriminationprocess unit 437 performs a discriminating process about a characterwith a complicated deletion line. In step SJ5, it does not recognize thecurrent character pattern as a character with a complicated deletionline.

The deletion line existence determining unit A shown in FIG. 95 showsthe processes in steps SJ4 and SJ5 indicating the contents of theprocess performed by the complicated deletion line characterdiscrimination process unit 437.

In step SJ41, the complexity amount extracting unit 443 extracts theamount of complexity such as an Euler number, line density, blackpicture element density, etc. to determine whether or not the currentcharacter pattern is a character with a complicated deletion line.

Then, in step SJ42, the complicated deletion line characterdiscriminating unit 444 determines using a predetermined thresholdwhether the object character pattern is a normal character or acharacter with a complicated deletion line.

For example, assuming that the threshold of the line density by thescanning in the horizontal direction is 2, the threshold of the linedensity by scanning in the vertical direction is 3, the threshold of anEuler number is −1, and the threshold of the black picture elementdensity is 0.6, it is determined that the character pattern is notprovided with a complicated deletion line when the line density by thescanning in the horizontal direction is equal to or smaller than 2, theline density by scanning in the vertical direction is equal to orsmaller than 3, an Euler number is equal to or larger than −1, and theblack picture element density is equal to or smaller than 0.6.Otherwise, the character pattern is discriminated as a character with acomplicated deletion line.

If the character pattern is discriminated as a character with acomplicated deletion line in step SJ43, a rejection code is output as arecognition result in step SJ5.

If the character pattern is not discriminated as a character with acomplicated deletion line, control is passed to step SJ6 as shown inFIG. 94, and the simple deletion line character discriminating unit 438performs a process of discriminating whether or not a horizontal line,that is, a single deletion line exists. If the pattern is discriminatedas a character with a deletion line, then a rejection code is output asa recognition result in step SJ7.

The discriminating process is indicated by the deletion line existencedetermining unit B in FIG. 95.

As shown in FIG. 95, the picture element histogram computing unit 445generates a horizontal direction contiguous projection histogram in stepSJ61.

A contiguous histogram can be, for example, obtained by adding blackpicture elements for every n=3 lines along the horizontal line with eachline shifted in the direction vertical to the horizontal simple deletionline as shown in FIG. 20. Therefore, even if the horizontal simpledeletion line is not exactly horizontal, it can be correctlydiscriminated as a simple deletion line.

If the histogram indicates the peak exceeding the threshold N in stepSJ62, the simple deletion line character candidate discriminating unit446 determines that there is a candidate for a deletion line in stepSJ63 and discriminates the pattern as a candidate for a character with asimple deletion line. If such a peak does not exist, it is determinedthat the pattern does not contain a candidate for a deletion line andthe pattern is discriminated as a normal character. If a character sizeis standardized in a pre-process, N can be a fixed value. If it is notstandardized in a pre-process, it is recommended that N is variabledepending on the width of the rectangle circumscribing the objectcharacter. In this case, the ratio of the threshold N to the width ofthe circumscribing rectangle should be appropriately given as a fixedvalue.

In step SJ63, if the simple deletion line character candidatediscriminating unit 446 discriminates the pattern as a candidate for acharacter with a deletion line, then control is passed to step SJ64, andthe character candidate storage unit 449 stores a candidate for acharacter with a deletion line (before removing a deletion line). Thedeletion line removing unit 450 detects and deletes a horizontal simpledeletion line of the candidate for a character with a deletion line.

A candidate for a deletion line is removed by an existing method, forexample, a method of extracting a line through an n line run length,etc.

The feature extracting process is performed by the identifying unit 439on a character pattern from which a deletion line has been removed instep S38. In step SJ9, the extracted feature is collated with thedictionary.

In the dictionary collation, the matching level is computed by referringto a given standard feature vector, and a character type with a featurevector indicating the highest similarity is output as a recognitionresult.

If it is determined that the collation result indicates rejection instep SJ65, the simple deletion line character determining unit 448determines that the candidate for a character with a deletion line is acharacter without a deletion line in step SJ66, transmits the originalimage of the candidate for a character with a deletion line temporarilystored in the character candidate storage unit 449 to the identifyingunit 439 which identifies,a character in step SJ67. The result outputdirecting unit 440 instructs the output unit 433 to output theidentification result.

If it is determined that the pattern has been identified as a characterafter the collating process in step SJ65, then control is passed to stepSJ68 and the simple deletion line character determining unit 448discriminates the candidate for a character with a deletion line as acharacter with a deletion line, and the result output directing unit 440outputs the recognition result as ‘REJECT’ to the output unit 433 instep SJ69.

The upper line in FIG. 71 shows an example of recognizing a candidatefor a deletion line about a candidate for a character with a simpledeletion line, identifying as ‘5’ the pattern stripped of the candidatefor a deletion line, and determining the pattern as a character with adeletion line. The lower line in FIG. 71 shows an example of recognizinga candidate for a deletion line in the candidate for a character with asimple deletion line, rejecting the pattern stripped of the candidatefor a deletion line, and therefore discriminating the candidate for theoriginal character with a simple deletion line as a normal character.

As described above, a determination is made about a character with acomplicated deletion line based on the amount of complexity includingall of the maximum line density in a predetermined direction, an Eulernumber, the black picture element density according to the presentembodiment.

Therefore, the present embodiment realizes the discrimination of acharacter with a complicated deletion line with high reliability at ahigh speed.

Furthermore, according to the present embodiment, a candidate for acharacter with a simple deletion line can be discriminated by countingthe number of picture elements in the contiguous projection method.

As a result, a candidate for a character with a deletion line includinga line drawn in a roughly horizontal direction as well as a predictedhorizontal simple deletion line can be easily and quickly discriminatedwith high reliability with a simple configuration.

As described above, misreading a character with a deletion line can bereduced by performing a deletion line existence determining processaccording to the aspect of the present embodiment. Additionally, acharacter deleted by a deletion line can be clearly rejected.

What is claimed is:
 1. An image recognition apparatus comprising: animage input unit that inputs an image, detects an image area whichcontains either of a character pattern without a deletion line image ora character pattern with deletion line image from the input image, andperforms a pre-process; a deletion line character discriminating unitthat determines whether a character pattern is at least one of a normalcharacter without a deletion line image, a character pattern with asimple deletion line image comprising a horizontal line image indicatingdeletion of a character and a character pattern with a complicateddeletion line image in a complicated form; and an identifying unit thatidentifies a character.
 2. The pattern recognizing apparatus accordingto claim 1, wherein said deletion line character discriminating unitcomprises: a complicated deletion line character discrimination processunit that discriminates a character pattern with a complicated deletionline image; and a simple deletion line character discriminating unitthat discriminates a character pattern, determined to have a simpledeletion line image, with a simple deletion line image on a candidatefor the character pattern based on whether an object from which thedeletion line image is removed can be identified as a character whensaid complicated deletion line discriminating unit does not discriminatea character pattern with a complicated deletion line image.
 3. Thepattern recognizing apparatus according to claim 1, wherein saiddeletion line character discriminating unit comprises: a character forminformation extracting unit that extracts, for a detected andpre-processed character pattern, character form information comprisingat least one of a complexity amount indicating the complexity ofgraphics and a picture element-number histogram obtained by a horizontaldirection contiguous projection in which black picture elements areadded in an object horizontal scanning line and a line in apredetermined range adjacent above and below an object horizontalscanning line; a discriminating unit that discriminates a candidate fora character pattern with a simple deletion line image or a candidate fora character pattern with a complicated deletion line image according tothe character form information extracted by said character forminformation extracting unit; and a deletion line character defining unitthat defines a character image with a simple deletion line image whensaid identifying unit can identify a character after removing a deletionline image from at least the candidate for a character pattern with asimple deletion line image.
 4. The pattern recognizing apparatusaccording to claim 2, wherein said complicated deletion line characterdiscrimination process unit comprises: a complexity extracting unit thatextracts an amount of complexity indicating the complexity of graphics;a complicated deletion line character discrimination unit thatdiscriminates a character pattern with a complicated deletion lineimage, and said simple deletion line character discriminating unitcomprises: a picture-element-number histogram computing unit thatcomputes a number of picture element by obtaining by a horizontaldirection contiguous projection in which black picture elements areadded in an object horizontal scanning line and a line in apredetermined range adjacent above and below an object horizontalscanning line; a simple deletion line character candidate discriminatingunit that discriminates a character pattern with a simple deletion lineimage candidate based on a computed number of picture elements; and asimple deletion line character defining unit that defines a characterpattern with a simple deletion line image when said identifying unitidentifies a character after removing a deletion line image from adiscriminated candidate for a character pattern with a simple deletionline image.
 5. The pattern recognizing apparatus according to claim 4,wherein said simple deletion line character defining unit comprises: adeletion line removing unit that removes a deletion line image from adiscriminated candidate for a character pattern with a simple deletionline image; a character candidate storage unit that stores a candidatefor a character pattern with a simple deletion line image beforeremoving the deletion line image; and a deletion line determining unitthat determines the candidate for a character pattern with a simpledeletion line image to be character with a simple deletion line imagewhen the candidate for a character pattern with a simple deletion lineimage after removing the deletion line image can be identified as acharacter, determines the candidate for a character pattern with asimple deletion line image stored in said character candidate storageunit as a normal character when the candidate for a character patternwith a simple deletion line image cannot be identified as a character,and transmits a determination result to said identifying unit.
 6. Animage recognition method comprising: inputting an image; detecting animage area which contains either of a character pattern without adeletion line image or a character pattern with deletion line image fromthe input image; performing a pre-process; discriminating, on acharacter pattern detected or pre-processed, whether a character patternis a normal character pattern without a deletion line image or acharacter with a deletion line image; discriminating whether thecharacter with a deletion line image is either a simple deletion lineimage comprising a horizontal line image indicating deletion of acharacter or complicated deletion line image in a complicated form; andidentifying the normal character without the deletion line image.
 7. Animage recognition method comprising: inputting an image; detecting animage area which contains either of a character pattern without adeletion line image or a character pattern with deletion line image fromthe input image; performing a pre-process; discriminating, on acharacter pattern detected or pre-processed, whether or not a characterpattern is provided with a complicated deletion line image;discriminating, when it is discriminated that the character is not acharacter pattern with the complicated deletion line image, whether thecharacter is the normal character pattern without the deletion lineimage or a character with a simple deletion line image comprising only ahorizontal line image indicating deletion of a character; andidentifying the normal character without the deletion line image.
 8. Apattern recognizing method comprising: inputting an image, detecting acharacter pattern from the input image and performing a pre-process;discriminating on a character pattern detected or pre-processed whetheror not a character pattern is provided with a complicated deletion lineimage character; computing on the character pattern detected orpre-processed, when the character pattern is not the character with acomplicated deletion line image, a picture element number histogram by ahorizontal direction contiguous projection in which black pictureelements are added in an object horizontal scanning line and a line in apredetermined range adjacent above and below an object horizontalscanning line; determining a candidate for a character pattern with asimple deletion line image based on the picture element histogram;defining a character pattern with a simple deletion line image when thecandidate for a character pattern with a simple deletion line image fromwhich the deletion line image is removed can be identified as acharacter, and defining a normal character when the candidate cannot beidentified as a character; and identifying the character for the definednormal character.
 9. The pattern recognizing method according to claim8, further comprising: when defining a character pattern with a simpledeletion line image when the candidate for a character pattern with asimple deletion line image from which the deletion line image is removedcan be identified as a character, and defining a normal character whenthe candidate cannot be identified as a character; determining whetheror not the character is a candidate for a character pattern with asimple deletion line image; removing the deletion line image when thecharacter is a candidate for a character pattern with a simple deletionline image; determining the character whether the candidate for acharacter pattern with a deletion line image from which the deletionline image has been removed can be identified as a character; defining acandidate for a character pattern with a simple deletion line image whenthe candidate can be identified as a character; defining a normalcharacter when the candidate cannot be identified as a character;determining whether or not the defined normal character can beidentified as a character in identifying the normal character;outputting a reject code when the normal character cannot be identifiedas a character; outputting a character type code when the normalcharacter can be identified as a character; and outputting the rejectcode when the character is provided with a deletion line.
 10. Acomputer-readable storage medium controlling a computer by: inputting animage; detecting an image area which contains either of a characterpattern without a deletion line image or a character pattern withdeletion line image from the input image; performing a pre-process;determining, using the character pattern detected or pre-processedwhether the character pattern is a normal character without a deletionline image or a character pattern with a deletion line image;determining whether the character pattern with a deletion line image isprovided with either a simple deletion line image comprising ahorizontal line image indicating deletion of a character or complicateddeletion lines image in a complicated form; and identifying thecharacter pattern without the deletion line image.
 11. A patternrecognizing apparatus comprising: image input means for inputting animage, detecting an image area which contains either of a characterpattern without a deletion line image or a character pattern withdeletion line image from the input image, and performing a pre-process;first deletion line character discriminating means for determiningwhether a character pattern is a normal character pattern without adeletion line image or a character pattern with deletion line image;second deletion line character discriminating means for determiningwhether the character pattern with a deletion line image is providedwith either a simple deletion line image comprising a horizontal lineimage indicating deletion of a character or complicated deletion linesimage in a complicated form; and identifying means for identifying acharacter.